You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "ddanielr (via GitHub)" <gi...@apache.org> on 2023/04/17 22:01:32 UTC

[GitHub] [accumulo-website] ddanielr opened a new pull request, #384: Add containerized development environment

ddanielr opened a new pull request, #384:
URL: https://github.com/apache/accumulo-website/pull/384

   Adds a dockerfile for building a containerized development environment. 
   Also adds the htmlproofer gem for use of validating html structure and links.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175931860


##########
README.md:
##########
@@ -108,6 +108,69 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.\

Review Comment:
   Paragraph flow just looks very cluttered to me. 
   I'm also unaware of any screen reader benefits but I'll switch these over anyway. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] keith-turner commented on pull request #384: Add containerized development environment

Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1516329035

   If I were to make a large change to the website where I needed local preview, I would use this new docker container.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175925626


##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   They are being copied so that the `RUN` command  on line 13 can execute the `bundle install` and install all the ruby dependencies in the container image. 
   
   This is done for repeatability and render speed. 
   
   The docker image is not attempting software installs as part of the render operation at runtime so we can guarantee that there are no render differences between machines, ensuring repeatable results. 
   
   Likewise, the render operation is the ONLY thing the container has to do, so it's quite fast and allows developers to work at speed.   
   
   There's also an added benefit that any gem updates with dependency issues should trigger failures in the docker build stage vs throwing an error during the render stage.
   
   This usage pattern is also laid out in the "How to use this image" section for the [base ruby image](https://hub.docker.com/_/ruby) 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175990270


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.

Review Comment:
   Not for the base container, but it provides an easy button for tagging. 
   
   If you run the validator build step separately, it will always build the first container or reference it from the cache. 
   This is due to the fact that they are in the same Dockerfile and the `base` container is an earlier stage.
    
   This build pattern is intended to ensure that the images don't drift from each other while also minimizing the number of Dockerfiles and the ordering of disjointed image builds. 
   
   I agree that the script is pretty small, but the only reason I kept the script is because we are tagging **both** images and it ensures the tagging is done correctly to minimize errors.  
   
   The `--tag` option on a docker build command only applies to the last built image in a Dockerfile. 
   That `--target` option limits the build step to only target the image declared as "base" in the Dockerfile. 
   `docker build --target base -t "$BASE_TAG" .` 
   
   If someone runs a build command without the `--target base` filter, the Dockerfile will build and tag the second image and will not tag the first image at all. Instead, that image just shows up as a docker image with no name and a SHA id.
   
   If that were to occur,  the dev would have a mis-tagged validator image, **and** an base image that had no referenced tag to easily figure out what it is. 
   
   Having a script to run and ensure images are tagged correctly would alleviate that issue while still allowing developers the freedom of manually running the build steps if they so chose.  
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175932956


##########
build-images.sh:
##########
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e 
+
+# This script is used to build and tag both docker images while still
+# keeping the images in the same Dockerfile. 
+# Doing this ensures that both images will have synced dependencies. 
+
+BASE_TAG="webdev"
+VALIDATE_TAG="${BASE_TAG}-validator"

Review Comment:
   I've gone ahead and removed this, but generally if I'm adding text to the bash variable, I like to use something to separate the variable name from the additional text. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1176010699


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > First, the name of the directory is "site", which is a bit confusing, because later, the site is built in `_site`. Would that location be `/site/_site/` then?
   
   Yes! 
   
   So three directories exist
   1. $PWD (root of the repository on the host)
   2. $PWD/_site (location of rendered changes on the host)
   3. /site (working directory of the container)
   
   These locations are mounted at container runtime to allow the webdev and webdev-validator containers to access the  source and rendered content. 
   
   I completely agree that the `/site` directory name is a bit overused. So I'm happy to take suggestions on a better name for the container side of things. 
   
   > Second, when is the rest of the website files copied over? Are those also copied over to the same `/site` directory inside the container? How does that work? Are they copied over to `/site` after the RUN commands but before the CMD commands?
   
   So everything that happens in the Dockerfile is part of the container build stage. 
   This is completely separate from actually running the container. 
   
   The `RUN` commands specified in this file are just for local actions we want to trigger as part of building a container. 
   CMD is the default command the container will execute when a `docker run` command is executed.
   
   The rest of the site is mounted as a read/write volume at runtime. The contents of `accumulo-website/` are mounted at the container's `/site` directory location for the webdev container.  
   
   Once the webdev container is running, and the source directory is mounted, then the CMD statement finally executes and the rendered content is created on the containers /site/_site directory, which maps to the host's $PWD/_site directory. 
   
   This results in the following dir structure: 
   
   webdev's container's filesystem
   ```
   /site
     - contents of accumulo-website
     - /_site 
        - Rendered html from jekyll 
   ```
   
   This ensures that no site files are embedded in a container and that the source repository remains the only location for changes to occur. 
   
   Now, we want to mount the rendered content into the validator container. However, the webdev-validator container needs to have a slightly modified Gemfile. 
   
   To accomplish this, the run command only mounts $PWD/_site to /site/_site. This sets up the following structure
   
   webdev-validator's filesystem
   ```
   /site
     - Gemfile (modified by container)
     - Gemfile.lock (modified by container)
     - /_site
        - Rendered html from jekyll
   ```
   
   > And third, just to clarify, the RUN command is using the Gemfiles to install the gems when the container image is first created and started (using `docker run`) and then the CMD commands are run. For subsequent stopping (`docker stop`) and restarting (`docker start`) , only the CMD commands are executed on startup? Is this correct?
   
   The RUN command is only triggered during a `docker build` command. 
   The container is never started during the `docker build` stage. 
   
   The CMD statement ONLY runs on a `docker run`  or `docker start` command and is never triggered during a `docker build` stage, only specified. 
   
   If it helps to think about it this way, CMD and ENTRYPOINT are like environment variables I am setting in my Dockerfile. I'm specifying the future behavior of my container when it runs. So, I can define those variables at any point in my Dockerfile, their placement in the Dockerfile has no bearing on what actions happen during container build. 
   
   Whereas RUN statements are actual, order dependent commands that are used to trigger immediate actions in my container during the build stage to modify the container in some way.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177174353


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > 2. $PWD/_site (location of rendered changes on the host)
   
   Gotcha. Rendered in the container process/memory space, but written to disk on the host because that is mounted.
   
   > I completely agree that the `/site` directory name is a bit overused. So I'm happy to take suggestions on a better name for the container side of things.
   
   Maybe "/mnt/repo" or "/mnt/staging" or "/mnt/build" or "/mnt/workdir"? I like putting things in `/mnt` because it makes things pretty clear that it's mounted, and from the perspective from inside the container, this is a pretty standard location for external filesystems to be mounted at.
   
   > So everything that happens in the Dockerfile is part of the container build stage. This is completely separate from actually running the container.
   
   My question isn't really about WHEN the RUN commands execute (I get that it's during the build stage), but WHERE (because it doesn't make sense that they would run locally on the host). My understanding is that the build stage involves running a container from the base image, executing RUN commands to modify the image, and when that build container stops, the resulting image is the modified image that we use to launch our own containers.
   
   Essentially:
   
   ```
   docker build -> create temporary build container from base image, start build container, exec RUN commands inside build container, stop build container, save modified image, delete build container
   docker create -> create container from image
   docker start -> run container with specified command, or stored CMD if command is unspecified
   docker run -> shorthand for docker create + docker start
   ```
   
   > This ensures that no site files are embedded in a container and that the source repository remains the only location for changes to occur.
   
   Some rendering might use /tmp inside the container. I'm not sure if any of that is preserved inside the container, but I don't think we care if it is or isn't.
   
   > Now, we want to mount the rendered content into the validator container. However, the webdev-validator container needs to have a slightly modified Gemfile.
   > 
   > To accomplish this, the run command only mounts $PWD/_site to /site/_site. This sets up the following structure
   
   My understanding here is that the build of the validator container does *not* have the repo mounted. So, the Gemfiles it is modifying are the ones copied to the base image when that was built.
   
   The confusion here is that when we run a container from the base image, we're mounting over the same directory as we copied these files. Are these just masked/hidden by the mount, then? Usually it's a good idea to mount to an empty directory. But in this case, it seems we're not doing that in the base image.
   
   If we're mounting over the copied Gemfiles, then there's no reason to create a second validator image... we can just modify the Gemfile during the base build, install *all* the gems to the image during the base image build. When we mount our website repo to render the Jekyll, we hide the modified Gemfiles. When we mount our rendered `_site` directory to do the validation, we re-expose the modified Gemfiles. We don't need a second image at all... we can do everything we want with the single image.
   
   Actually, we don't even need to copy over the Gemfiles during the build stage... those aren't needed for the htmlproofer. htmlproofer has need of its own Gemfiles. We're only copying over the ones from the repo and modifying them... but we don't need to do that... we can just have a separate set for the htmlproofer... and we don't even need the lock one at all for that.
   
   I think we can simplify things dramatically by just using one container.
   
   I do have one more question about WORKDIR... it seems that's overloaded to specify the current working directory inside the build container as well as the runtime container. But, I don't see why that necessarily needs to be the case.
   
   Also, one thing I was thinking was that it can be really hard to verify links using the rendered HTML, because you don't have an absolute directory to resolve absolute paths. The generated site is intended to be served at the root of a webserver. I don't know if htmlproofer supports this, but it'd probably be better to run the validation against `http://localhost:4000/` rather than `file:///site/_site/`.
   
   If we want to get this in quickly, we could get the Dockerfile in without the validator stuff right away. I think that because the validator stuff adds so much complexity (extra steps, extra documentation, maybe extra containers) that still needs polishing, I want to go back to my much earlier opinion that the validation stuff should be added in a separate PR. At this point, I think it's likely to hold up this PR further.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177176676


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.

Review Comment:
   I think my point here is moot now... since I think we can do everything we need with a single image. (See other comment)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1176010699


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > First, the name of the directory is "site", which is a bit confusing, because later, the site is built in `_site`. Would that location be `/site/_site/` then?
   
   Yes! 
   
   So three directories exist
   1. $PWD (root of the repository on the host)
   2. $PWD/_site (location of rendered changes on the host)
   3. /site (working directory of the container)
   
   These locations are mounted at container runtime to allow the webdev and webdev-validator containers to access the  source and rendered content. 
   
   I completely agree that the `/site` directory name is a bit overused. So I'm happy to take suggestions on a better name for the container side of things. 
   
   > Second, when is the rest of the website files copied over? Are those also copied over to the same `/site` directory inside the container? How does that work? Are they copied over to `/site` after the RUN commands but before the CMD commands?
   
   So everything that happens in the Dockerfile is part of the container build stage. 
   This is completely separate from actually running the container. 
   
   The `RUN` commands specified in this file are just for local actions we want to trigger as part of building a container. 
   CMD is the default command the container will execute when a `docker run` command is executed.
   
   The rest of the site is mounted as a read/write volume at runtime. The contents of `accumulo-website/` are mounted at the container's `/site` directory location for the webdev container.  
   
   Once the webdev container is running, and the source directory is mounted, then the CMD statement finally executes and the rendered content is created on the containers /site/_site directory, which maps to the host's $PWD/_site directory. 
   
   This results in the following dir structure: 
   
   webdev's container's filesystem
   /site
     - contents of accumulo-website
     - /_site 
        - Rendered html from jekyll 
   
   This ensures that no site files are embedded in a container and that the source repository remains the only location for changes to occur. 
   
   Now, we want to mount the rendered content into the validator container. However, the webdev-validator container needs to have a slightly modified Gemfile. 
   
   To accomplish this, the run command only mounts $PWD/_site to /site/_site. This sets up the following structure
   
   webdev-validator's filesystem
   /site
     - Gemfile (modified by container)
     - Gemfile.lock (modified by container)
     - /_site
        - Rendered html from jekyll
   
   > And third, just to clarify, the RUN command is using the Gemfiles to install the gems when the container image is first created and started (using `docker run`) and then the CMD commands are run. For subsequent stopping (`docker stop`) and restarting (`docker start`) , only the CMD commands are executed on startup? Is this correct?
   
   The RUN command is only triggered during a `docker build` command. 
   The container is never started during the `docker build` stage. 
   
   The CMD statement ONLY runs on a `docker run`  or `docker start` command and is never triggered during a `docker build` stage, only specified. 
   
   If it helps to think about it this way, CMD and ENTRYPOINT are like environment variables I am setting in my Dockerfile. I'm specifying the future behavior of my container when it runs. So, I can define those variables at any point in my Dockerfile, their placement in the Dockerfile has no bearing on what actions happen during container build. 
   
   Whereas RUN statements are actual, order dependent commands that are used to trigger immediate actions in my container during the build stage to modify the container in some way.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178153859


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.
+
+The webdev container does not store any site content inside of it.
+For rendering to function, the local directory must be mounted inside
+the container in a read/write configuration.
+
+This mounting is accomplished with the volume `-v` flag in the
+following `docker run` command.

Review Comment:
   I assumed this comment was replacing lines 127-132. So that has now been modified. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1176459128


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.

Review Comment:
   Change has been included.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177332360


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > always run in a docker image
   
   I think part of my earlier confusion comes from your loose interchange of "image" and "container". I think I know enough to parse those out without trouble now. But, for clarity, I don't think it makes sense to say anything is running in an image. I think if we were to be precise, then things run inside containers (as opposed to images), and containers are launched/created from an image. I think of images as just snapshots of a particular container's (stopped) state, from which another container can be launched. The build stage, therefore, just seems to be a starting up a container with one base image as input, and a new (possibly incremental CoW) image as output.
   
   > The final image layer generated is the last "layer" of changes that comprise the image and is tagged as the final image.
   > 
   > So it's different from building a virtual machine image where you only get an single output of a image based on all the commands run within the image at build.
   > 
   > This allows Docker to share image layers between containers and save space.
   
   Yeah, that just seems to be a basic copy-on-write incremental filesystem implementation. Not fundamentally different than qcow files and libvirtd... just a lot of easier-to-use tooling around those same fundamental concepts (and the primary fact that isolation comes from OS containerization features instead of virtualization for machine emulation). You still have to have a running container in which the build is executed. The docker documentation at https://docs.docker.com/engine/reference/commandline/build/ specifically refers to "the containers used in the build" and there are flags you can put on the command line to customize the cgroups for this running build container. So, there is actually a container running in the build stage, according to the docs... just not the same container you make later from the image that is the output of the build stage.
   
   
   > The github action only referenced publishing the contents of `_site` so I don't think the possible use of `/tmp` matters since it was never published before.
   
   Yeah, it shouldn't matter for anything we're doing... I'm just not sure how Docker might handle that... whether it would preserve those out-of-tree temporary file states or not.
   
   > So WORKDIR sets the PWD for any following RUN, COPY, ENTRYPOINT, and CMD commands in the Dockerfile.
   
   This is what I was noticing. Specifically, we're relying on it being the same for RUN and CMD, even though those run at completely separate phases. I was thinking that we might want to be in one directory for building the image, and another directory when we run the container. Not sure if there's a way to control that with that much granularity. Can always `(cd /somewhere else && dosomething)`, I suppose.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177328415


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > Maybe "/mnt/repo" or "/mnt/staging" or "/mnt/build" or "/mnt/workdir"? I like putting things in `/mnt` because it makes things pretty clear that it's mounted, and from the perspective from inside the container, this is a pretty standard location for external filesystems to be mounted at.
   
   Out of those options I like `/mnt/workdir` the best. 
   
   > 
   > > So everything that happens in the Dockerfile is part of the container build stage. This is completely separate from actually running the container.
   > 
   > My question isn't really about WHEN the RUN commands execute (I get that it's during the build stage), but WHERE (because it doesn't make sense that they would run locally on the host). My understanding is that the build stage involves running a container from the base image, executing RUN commands to modify the image, and when that build container stops, the resulting image is the modified image that we use to launch our own containers.
   > 
   > Essentially:
   > 
   > ```
   > docker build -> create temporary build container from base image, start build container, exec RUN commands 
   > inside build container, stop build container, save modified image, delete build container
   > docker create -> create container from image
   > docker start -> run container with specified command, or stored CMD if command is unspecified
   > docker run -> shorthand for docker create + docker start
   >```
   
   RUN commands are always run in a docker image and are executed in the WORKDIR that has been specified in the Dockerfile. 
   https://docs.docker.com/engine/reference/builder/#run 
   
   For now, realize that you don't have a container "running" as part of the build process.
   Instead you have something similar to a git tree where each command in a dockerfile is executed by the docker deamon and results in a new commit layer being added (ENV, LABEL, and WORKDIR are special and don't create a new commit layer).  That new layer is then used by the next command as it's container context.
   
   https://docs.docker.com/build/building/packaging/#dockerfile 
   
   The final image layer generated is the last "layer" of changes that comprise the image and is tagged as the final image.
   
   So it's different from building a virtual machine image where you only get an single output of a image based on all the commands run within the image at build.
   
   This allows Docker to share image layers between containers and save space. 
   
   > Some rendering might use /tmp inside the container. I'm not sure if any of that is preserved inside the container, but I don't think we care if it is or isn't.
   
   The github action only referenced publishing the contents of `_site` so I don't think the possible use of `/tmp` matters since it was never published before.
   https://github.com/apache/accumulo-website/blob/0d3904b0c3b4eb839bb59158946d9068fba7ef53/.github/workflows/jekyll.yaml#L51-L56
   
   
   > > Now, we want to mount the rendered content into the validator container. However, the webdev-validator container needs to have a slightly modified Gemfile.
   > > To accomplish this, the run command only mounts $PWD/_site to /site/_site. This sets up the following structure
   > 
   > My understanding here is that the build of the validator container does _not_ have the repo mounted. So, the Gemfiles it is modifying are the ones copied to the base image when that was built.
   
   Correct. 
   
   > 
   > The confusion here is that when we run a container from the base image, we're mounting over the same directory as we copied these files. Are these just masked/hidden by the mount, then? Usually it's a good idea to mount to an empty directory. But in this case, it seems we're not doing that in the base image.
   
   I agree that it's a good idea to mount to an empty directory for traditional systems. 
   
   However, mapping an existing directory overtop of a pre-defined one is a standard action with containers. 
   Docker just discards the contents of the container directory and references the host dir.
   
   > 
   > If we're mounting over the copied Gemfiles, then there's no reason to create a second validator image... we can just modify the Gemfile during the base build, install _all_ the gems to the image during the base image build. When we mount our website repo to render the Jekyll, we hide the modified Gemfiles. When we mount our rendered `_site` directory to do the validation, we re-expose the modified Gemfiles. We don't need a second image at all... we can do everything we want with the single image.
   > 
   > Actually, we don't even need to copy over the Gemfiles during the build stage... those aren't needed for the htmlproofer. htmlproofer has need of its own Gemfiles. We're only copying over the ones from the repo and modifying them... but we don't need to do that... we can just have a separate set for the htmlproofer... and we don't even need the lock one at all for that.
   > 
   > I think we can simplify things dramatically by just using one container.
   
   My original design had the single container but I had split it into two to restrict the Gemfile changes. 
   So if there's a better way to separate Gemfile changes I'm happy to try that. 
   I do agree that it massively simplifies things. 
   
   > 
   > I do have one more question about WORKDIR... it seems that's overloaded to specify the current working directory inside the build container as well as the runtime container. But, I don't see why that necessarily needs to be the case.
   
   I'm not exactly sure what question you're asking here, but I'll try and provide the information I know. 
    
   So WORKDIR sets the PWD for any following RUN, COPY, ENTRYPOINT, and CMD commands in the Dockerfile. 
   
   If you don't set WORKDIR explicitly then it will default to `/` if it was not previously defined by the image in your FROM command.
    
   Typically you would set the WORKDIR to ensure that all of your operations are running from an intended place, vs happening in unexpected locations within the container. 
   
   If you wanted your runtime container to run things in a different location than the build commands, you can simply add another WORKDIR line in your Dockerfile and any commands defined after it will use the new location (like CMD). 
   
   https://docs.docker.com/engine/reference/builder/#workdir 
   
   > 
   > Also, one thing I was thinking was that it can be really hard to verify links using the rendered HTML, because you don't have an absolute directory to resolve absolute paths. The generated site is intended to be served at the root of a webserver. I don't know if htmlproofer supports this, but it'd probably be better to run the validation against `http://localhost:4000/` rather than `file:///site/_site/`.
   > 
   htmlproofer seems explicitly designed to run against rendered HTML code in a directory. I didn't see anything in the documentation related to pointing it at a live site.
   The project does support [adjusting for a baseurl](https://github.com/gjtorikian/html-proofer#adjusting-for-a-baseurl)
   
   > If we want to get this in quickly, we could get the Dockerfile in without the validator stuff right away. I think that because the validator stuff adds so much complexity (extra steps, extra documentation, maybe extra containers) that still needs polishing, I want to go back to my much earlier opinion that the validation stuff should be added in a separate PR. At this point, I think it's likely to hold up this PR further.
   
   At this point I agree. 
   
   Figuring out how to support gem isolation is important but we can get the base image docker changes in and provide benefit for any developers generating documentation. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175920336


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+A containerized development environment can be built using the local Dockerfile. 
+
+```bash
+docker build -t webdev .
+```
+
+The webdev container's run command will run jekyll's serve command with the 
+polling option enabled.  This allows for immediate review of rendered changes.   

Review Comment:
   Markdown's behavior with the trailing spaces is confusing a bit, I think, because it's not clear if a newline is intended or not. I'm glad there's an alternative syntax for it, but I also don't think we need line breaks in these paragraphs. If the break is important, it's probably worth creating a new paragraph.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178104394


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > This is what I was noticing. Specifically, we're relying on it being the same for RUN and CMD, even though those run at completely separate phases. I was thinking that we might want to be in one directory for building the image, and another directory when we run the container. Not sure if there's a way to control that with that much granularity. Can always `(cd /somewhere else && dosomething)`, I suppose.
   
   I don't know what benefit you get from being in a different directory that would justify the change.
   Overall this Dockerfile is pretty simple. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1523685794

   Are there any more requested changes to this code? Or can this be merged?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1172024158


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+A containerized development environment can be built using the local Dockerfile. 
+
+```bash
+docker build -t webdev .
+```
+
+The webdev container's run command will run jekyll's serve command with the 
+polling option enabled.  This allows for immediate review of rendered changes.   

Review Comment:
   Consecutive spaces get collapsed in rendered Markdown/HTML, so the double space after a period can be removed. Trailing spaces throughout our docs should also be removed. I've suggested changes to this one line, but it should be addressed everywhere in new PRs.
   
   ```suggestion
   polling option enabled. This allows for immediate review of rendered changes.
   ```



##########
Gemfile:
##########
@@ -2,5 +2,6 @@ ruby '>=2.5.1'
 source 'https://rubygems.org'
 gem 'jekyll', '>= 4.2.0'
 gem 'jekyll-redirect-from', '>= 0.16.0'
+gem 'html-proofer', '>= 4.4.0'

Review Comment:
   A comment should be added here, to note that html-proofer is not used for the build, but just specified so the optional link checking step can be manually executed.



##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+A containerized development environment can be built using the local Dockerfile. 
+
+```bash
+docker build -t webdev .
+```
+
+The webdev container's run command will run jekyll's serve command with the 
+polling option enabled.  This allows for immediate review of rendered changes.   
+
+```bash
+docker run -d -v $PWD:/site -p 4000:4000 --name accumulo-website webdev

Review Comment:
   I'm not sure if `$PWD` is guaranteed to be set or if it always returns what you want it to (it can contain relative paths, I think)... but it definitely could contain spaces, so quoting it is necessary. I'm making this suggestion once, but it should be fixed everywhere.
   
   ```suggestion
   docker run -d -v "$PWD":/site -p 4000:4000 --name accumulo-website webdev
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178152587


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.
+
+The webdev container does not store any site content inside of it.
+For rendering to function, the local directory must be mounted inside
+the container in a read/write configuration.
+
+This mounting is accomplished with the volume `-v` flag in the
+following `docker run` command.
+
+```bash
+docker run -d -v "$PWD":/mnt/workdir -p 4000:4000 webdev
+```
+
+This provides the ability to immediately review rendered content changes at
+[http://127.0.0.1:4000/](http://127.0.0.1:4000/).
+

Review Comment:
   Changes Added!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175922007


##########
build-images.sh:
##########


Review Comment:
   I figured there might be a need to keep the Dockerfile at the root. That's fine.
   As for shellcheck, I thought it was available in EPEL for CentOS. It's written in Haskell, so maintaining its dependencies might be a problem for some versions of CentOS. It might also be available as a snap. But, another option is to just paste the contents into https://www.shellcheck.net/ if you don't want to install it. In the main Accumulo repo, we have a GitHub Actions check for it. I haven't gotten around to adding that to our other repos.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175980499


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.
+
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.

Review Comment:
   Yes the mounting is being done by the `-v` option. 
   Line 131 should be the exact command someone needs to run to render their changes locally with the container. 
   
   Mount is a bit more verbose with the only requirement that the target directory must exist prior to the mount operation while `-v` has no such requirement. Since we control the creation of the container, we would just need to create the target directories as part of our build steps. 
   
    If we swapped the commands out, the new one would look like this: 
   ` docker run -d --mount type=bind,source="$pwd",target=/site -p 4000:4000 webdev`
   vs `docker run -d -v "$pwd":/site -p 4000:4000 webdev`
   
   Personally I prefer the shorthand command, but I'll defer if  using `--mount` helps relate things together. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177180860


##########
Dockerfile:
##########
@@ -0,0 +1,43 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+# Gems will be installed under GEM_HOME which is set by the ruby image.
+# See https://hub.docker.com/_/ruby for details. 
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup
+
+ENV HOST=0.0.0.0
+ENV PORT=4000
+
+EXPOSE $PORT
+
+CMD bundle exec jekyll serve --force-polling -H $HOST -P $PORT

Review Comment:
   ```suggestion
   # Configure the default command to build from the mounted repository
   CMD bundle exec jekyll serve --force-polling -H $HOST -P $PORT
   ```



##########
Dockerfile:
##########
@@ -0,0 +1,43 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+# Gems will be installed under GEM_HOME which is set by the ruby image.
+# See https://hub.docker.com/_/ruby for details. 
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+

Review Comment:
   ```suggestion
   # Copy over the Gemfiles so that all build dependencies are installed
   # during the docker build. At runtime, these will be available to Jekyll
   # from the mounted directory. But that's not available during the
   # docker build, so we need to copy them in to pre-install the Gems
   COPY Gemfile /site/Gemfile
   COPY Gemfile.lock /site/Gemfile.lock
   
   # Gems will be installed under GEM_HOME which is set by the ruby image.
   # See https://hub.docker.com/_/ruby for details. 
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178142196


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > I don't know what benefit you get from being in a different directory that would justify the change.
   
   Mostly, clarity in understanding which file was being edited (the copy or the mounted original). But I agree this doesn't matter much.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175800039


##########
Dockerfile:
##########


Review Comment:
   I think this Dockerfile could use some comments to explain its intended purpose as a short summary.



##########
build-images.sh:
##########
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e 
+
+# This script is used to build and tag both docker images while still
+# keeping the images in the same Dockerfile. 
+# Doing this ensures that both images will have synced dependencies. 
+
+BASE_TAG="webdev"
+VALIDATE_TAG="${BASE_TAG}-validator"
+
+docker build --target base -t ${BASE_TAG} .
+docker build -t ${VALIDATE_TAG} .

Review Comment:
   Quotes are needed here. Curly braces aren't.
   
   ```suggestion
   docker build --target base -t "$BASE_TAG" .
   docker build -t "$VALIDATE_TAG" .
   ```



##########
build-images.sh:
##########


Review Comment:
   This file could be put in the `_scripts/` directory and the first line could be modified to conform to the same standard `#! /usr/bin/env bash`. I'm not sure if there's a better place for the Dockerfile, too, but I do know we have a lot of clutter at the root, so if there is a possibility of a better location, that might be good too. If there isn't a better location, I think the Dockerfile can stay at the root, but I think this could still be moved to the `_scripts/` directory. It would also be good to use `shellcheck` to test the script for quality, and maybe even `shfmt` (maybe `shfmt -ln bash -l -d -i 2 -ci -s .` like the main accumulo repo) to verify some standardized formatting (but this is not strictly necessary... especially since this is a very small script).



##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   Is there a one-line comment that could be added to explain why these are being copied? It's not clear to me, since this is not a normal step when building locally.



##########
README.md:
##########
@@ -108,6 +108,69 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.\

Review Comment:
   I see you're using a lot of backslashes here in the README to add arbitrary line breaks. Micro-managing the flow like that is a very different style of writing from our usual Markdown. I recommend just sticking to regular paragraph flow, and let the browser handle line wrapping automatically for long lines. If a sentence needs to stand out on its own, rather than just using a line break, just put a blank line between to make a new new paragraph. My understanding is that separate paragraphs are also better for some screen readers for people with low vision, as well, but I could be wrong about that. At the very least, that would be more consistent with our existing style, and a lot easier to maintain.
   
   ```suggestion
   Dockerfile.
   
   ```



##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup

Review Comment:
   I'm not super familiar with the base ruby image, but in my environment, the system gems are provided by system RPMs in my distribution. This has been my experience using CentOS/RHEL/Fedora. So, a `yum` (or `dnf`) update is normally sufficient and I don't need to do a `gem update`. Is the situation different for the base ruby image?
   
   Also, the instructions on the README in https://github.com/apache/accumulo-website#local-builds-for-testing explain in detail how to set up the GEM environment as a non-supervisor user for local gems installed to the user directory by bundler. I think it would be helpful if this Dockerfile did something similar, just automated, so instead of setting things up differently by doing system installed gems. That way, there's only a single set of instructions to follow, whether we're doing something via a local build, or trying to maintain this Dockerfile in future. However, it looks like this is relying on a different set of environmental assumptions from the those in the README. It looks like this is installing gems using the superuser, and gems are being installed to a system directory rather than a local user directory. I'm not 100% certain of this, since I'm not super familiar with Docker. If this is diverging from the detailed README instructions, it would be helpful to have some 
 inline comments explaining what's being done here, so we have a better chance of being able to maintain it going forward.



##########
build-images.sh:
##########
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e 
+
+# This script is used to build and tag both docker images while still
+# keeping the images in the same Dockerfile. 
+# Doing this ensures that both images will have synced dependencies. 
+
+BASE_TAG="webdev"
+VALIDATE_TAG="${BASE_TAG}-validator"

Review Comment:
   Curly braces aren't needed here, but it's not hurting, so it's up to you.
   
   ```suggestion
   VALIDATE_TAG="$BASE_TAG-validator"
   ```



##########
README.md:
##########
@@ -108,6 +108,69 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.\
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.\
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.\
+This provides the ability to immediately review rendered content changes.

Review Comment:
   ```suggestion
   This action will produce two containers: `webdev` and `webdev-validator`.
   The webdev container will execute a `jekyll serve` command with the
   polling option enabled.
   This provides the ability to immediately review rendered content changes.
   ```



##########
README.md:
##########
@@ -108,6 +108,69 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.\
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.\
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.\
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.\
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.

Review Comment:
   ```suggestion
   Shell access can be obtained by overriding the default container command.
   This is useful for adding new gems, or modifying the Gemfile.lock for updating
   existing dependencies.
   ```



##########
README.md:
##########
@@ -108,6 +108,69 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.\
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.\
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.\
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.\
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.
+
+#### Validation environment
+
+The `webdev-validator` image can be used to run validation tests against the
+rendered website content.\
+The output directory `_site` needs to be volume mounted for these tests to work.

Review Comment:
   ```suggestion
   The `webdev-validator` image can be used to run validation tests against the
   rendered website content.
   The output directory `_site` needs to be volume mounted for these tests to work.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175893463


##########
build-images.sh:
##########


Review Comment:
   I moved the  `build-images.sh` script into `_scripts` and swapped to the `env` line. 
   
   `shellcheck` looks to not be an easy install on centos, but I added shfmt via `go install` and formatted the script against your command.  
   
   Unfortunately, I don't think we can move the Dockerfile out from the root directory location. 
   
   When docker builds an image, the current directory of the Dockerfile is passed as the build context to the docker daemon. This restricts directory transversal and file references above the build context do not work .
   
   Since the Gemfile and Gemfile.lock are needed by the build operation, the Dockerfile has to exist at the same directory level or higher. 
   
   Ruby wants the Gemfile at the project's root so I think that's at an impasse.  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175783412


##########
Gemfile:
##########
@@ -2,5 +2,6 @@ ruby '>=2.5.1'
 source 'https://rubygems.org'
 gem 'jekyll', '>= 4.2.0'
 gem 'jekyll-redirect-from', '>= 0.16.0'
+gem 'html-proofer', '>= 4.4.0'

Review Comment:
   This is OBE as the html-proofer gem has been moved into a defined install step in the validator container. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175943829


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.

Review Comment:
   I think this would expose the rendered site at `http://127.0.0.1:4000/` ?
   
   ```suggestion
   This provides the ability to immediately review rendered content changes at http://127.0.0.1:4000/ .
   ```



##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup
+
+ENV HOST=0.0.0.0
+ENV PORT=4000
+
+EXPOSE $PORT
+
+CMD bundle exec jekyll serve --force-polling -H $HOST -P $PORT
+
+# Create a separate validation container for testing tools that
+# can be used to validate the rendered HTML code.
+
+FROM base

Review Comment:
   Should `base` be given a more specific name, in case somebody's Docker environment already has a container with that name, or does Docker already provide enough disambiguation for the name of running containers?



##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   I'm a little confused by the name of this directory and the comment here.
   
   First, the name of the directory is "site", which is a bit confusing, because later, the site is built in `_site`. Would that location be `/site/_site/` then?
   
   Second, when is the rest of the website files copied over? Are those also copied over to the same `/site` directory inside the container? How does that work? Are they copied over to `/site` after the RUN commands but before the CMD commands?
   
   And third, just to clarify, the RUN command is using the Gemfiles to install the gems when the container image is first created and started (using `docker run`) and then the CMD commands are run. For subsequent stopping (`docker stop`) and restarting (`docker start`) , only the CMD commands are executed on startup? Is this correct?



##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.

Review Comment:
   Do we always need to create both?
   
   I know I already provided feedback on the script, but given how small the `build-images.sh` script is, it might make sense to just inline the two build commands into the README, so that the second one is optional, and get rid of the script. It's basically a two line script, and the second one is optional if you're not running the validator.



##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.
+
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.

Review Comment:
   Is there a simple command to do that mounting that we can add to this README, or is that what is being done by the `-v` command flag? In trying to figure this out, I saw https://docs.docker.com/storage/bind-mounts/, which recommends using `--mount` instead.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178144454


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.

Review Comment:
   Ah I missed that in my earlier removal. 
   Thanks for the catch! 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1515701644

   # Container use
   
   I wouldn't discount the use of a containerized environment for doing website changes as I've had conversations with at least two accumulo devs who plan on using it besides myself.
   
   Overall I've found that build environments that can be quickly rebuilt and shared helps reduce "works on my machine" issues and update rot.  
   
   # Local Dev issues 
   I chose a source container that matches the current ruby version with the hope that there were minimal changes between the containerized environment and any bare metal development environments.
   
   The gem lock updates were mostly generated from the containerized version, however I'm happy to spend more time seeing if I can eliminate those differences. 
   
   # html validation
   
   I explicitly did not add the htmlproofer to the GitHub build step as it currently finds thousands of html linking errors in the current site. 
   
   A large majority of these findings are valid
   and I did not want to add a failing build step that would block other documentation work. 
   
   Instead, adding it as a manual run option to allow for partial fixes seemed the best approach.
   
   However, I can split the validate work into a secondary container that uses a separate gem file and should reduce the possibility of conflicts with a local dev environment. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1176458521


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.
+
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.

Review Comment:
   Ok, I've rewritten the README to include more direct wording regarding mounting the local directory for both the shell access command, and the original webdev run command. 
   
   Let me know what you think!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1176463077


##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup

Review Comment:
   Comment has been added



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175782064


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+A containerized development environment can be built using the local Dockerfile. 
+
+```bash
+docker build -t webdev .
+```
+
+The webdev container's run command will run jekyll's serve command with the 
+polling option enabled.  This allows for immediate review of rendered changes.   

Review Comment:
   Markdown renders double spaces after a line as a line break. 
   I've gone through and swapped out the double spaces for the `\` character instead. 
   This should still render a line break in markdown while being a bit nicer to view. 
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175782421


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+A containerized development environment can be built using the local Dockerfile. 
+
+```bash
+docker build -t webdev .
+```
+
+The webdev container's run command will run jekyll's serve command with the 
+polling option enabled.  This allows for immediate review of rendered changes.   
+
+```bash
+docker run -d -v $PWD:/site -p 4000:4000 --name accumulo-website webdev

Review Comment:
   I've implemented the change and tested that the quoted commands still work as intended. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175893463


##########
build-images.sh:
##########


Review Comment:
   I moved the  `build-images.sh` script into `_scripts` and swapped to the `env` line. 
   
   `shellcheck` looks to not be an easy install on centos, but I added shfmt via `go install` and formatted the script against your command.  
   
   Unfortunately, I don't think we can move the Dockerfile out from the root directory location. 
   
   When docker builds an image, the current directory of the Dockerfile is passed as the build context to the docker daemon. This restricts directory transversal and file references above the build context do not work.
   
   Since the Gemfile and Gemfile.lock are needed by the build operation, the Dockerfile has to exist at the same directory level or higher. 
   
   Ruby wants the Gemfile at the project's root so I think that's at an impasse.  



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175935000


##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup

Review Comment:
   > Docker is operating on it's own separate filesystem layer so nothing installed in the container will affect anything on the given system. This accomplishes the same separation as setting `GEM_HOME` to a user directory.
   
   My main concern isn't separation between the container filesystem and the local filesystem, but the user being able to comprehend the analogous behavior inside the container to the local system. Specifically, in the local system, there is a separation between the OS's package manager and the gem package management by setting `GEM_HOME` so that the gem package manager stores stuff in a different location than the locations where RPM or DEB packages get installed. This separation is comprehensible in the README because `GEM_HOME` is explicitly set. It's not clear how this separation occurs inside the container.
   
   > 
   > If you're interested, the [Ruby image spec](https://hub.docker.com/_/ruby) calls out various environment variables that are set in the image, including `GEM_HOME` which is set to `/usr/local/bundle`.
   
   Perfect. In that case, I think all that's needed to help connect the behavior of this container to the instructions in the README is to add a simple comment above the gem commands that mention something like: `GEM_HOME is set by the container`. You probably don't even need to specify that it's `/usr/local/bundle`, because that could change. Just knowing that it's already set should be enough to connect one's understanding of the README to what's going on in the container.
   
   > 
   > The explicit `gem install bundler` command isn't needed as the Gemfile.lock requires Ruby 2.7 to be the minimum installed and bundler was added as a default gem in 2.6.
   
   Not really relevant for this PR, but just to highlight this challenge for some: Annoyingly, in Fedora 37, even though ruby.rpm is 3.1 and rubygems.rpm is 3.3, rubygems-bundler.rpm still ships in a separate optional RPM, even though it's supposed to be a default gem since 2.6 and should be included with rubygems.rpm. I think there was a discussion about fixing this, but I lost track of the discussion.
   
   > 
   > So, I don't see this diverging from the detailed readme instructions as `GEM_HOME` is already set in the container and the next command to run is `bundle install`.
   
   Now that I understand what it's doing (thanks for digging in and explaining it), I agree it's not different. I think just a simple comment can help connect the dots for future devs trying to figure this out.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178125321


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment

Review Comment:
   We can get rid of this section header, since we don't need to go that deep and there's only one section here now. We can revisit later, if needed.
   
   ```suggestion
   ```



##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.

Review Comment:
   ```suggestion
   A containerized development environment can be built using the local
   Dockerfile. You can build it with the following command:
   
   ```



##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.
+
+The webdev container does not store any site content inside of it.
+For rendering to function, the local directory must be mounted inside
+the container in a read/write configuration.
+
+This mounting is accomplished with the volume `-v` flag in the
+following `docker run` command.

Review Comment:
   When you run a container using the webdev image, your current working
   directory will be mounted, so that any changes made by the build inside
   the container will be reflected in your local workspace. This is done with the
   `-v` flag. To run the container, execute the following command:



##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.

Review Comment:
   ```suggestion
   This action will produce a `webdev` image, with all the website's build
   prerequisites preinstalled. When a container is run from this image, it
   will perform a `jekyll serve` command with the polling option enabled,
   so that changes you make locally will be immediately reflected.
   ```



##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.
+
+The webdev container does not store any site content inside of it.
+For rendering to function, the local directory must be mounted inside
+the container in a read/write configuration.
+
+This mounting is accomplished with the volume `-v` flag in the
+following `docker run` command.
+
+```bash
+docker run -d -v "$PWD":/mnt/workdir -p 4000:4000 webdev
+```
+
+This provides the ability to immediately review rendered content changes at
+[http://127.0.0.1:4000/](http://127.0.0.1:4000/).
+

Review Comment:
   ```suggestion
   While this container is running, you will be able to review the rendered website
   in your local browser at [http://127.0.0.1:4000/](http://127.0.0.1:4000/).
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1170054697


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Docker dev environment 

Review Comment:
   Suggestion added in b1547a3



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175974432


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup
+
+ENV HOST=0.0.0.0
+ENV PORT=4000
+
+EXPOSE $PORT
+
+CMD bundle exec jekyll serve --force-polling -H $HOST -P $PORT
+
+# Create a separate validation container for testing tools that
+# can be used to validate the rendered HTML code.
+
+FROM base

Review Comment:
   base is only relative to this Dockerfile since we've defined what `base` means earlier in the file. 
   `FROM ruby:2.7.8-slim-bullseye as base` 
   
   Think of it as a locally scoped variable that will cease to exist once the `docker build` command completes. 
   We could make it more explicit like `ruby-base` or `dev-base` but that doesn't provide benefit outside of better naming. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177332360


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > always run in a docker image
   
   I think part of my earlier confusion comes from your loose interchange of "image" and "container". I think I know enough to parse those out without trouble now. But, for clarity, I don't think it makes sense to say anything is running in an image. I think if we were to be precise, then things run inside containers (as opposed to images), and containers are launched/created from an image. I think of images as just snapshots of a particular container's (stopped) state, from which another container can be launched. The build stage, therefore, just seems to be a starting up a container with one base image as input, and a new (possibly incremental CoW) image as output.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177332360


##########
Dockerfile:
##########
@@ -0,0 +1,40 @@
+# This Dockerfile builds an ruby environment for jekyll that empowers
+# making updates to the accumulo website without requiring the dev
+# to maintain a local ruby development environment.
+
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+
+# Copy over the Gemfiles so that all build dependencies are installed
+# during build vs at runtime.
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock

Review Comment:
   > always run in a docker image
   
   I think part of my earlier confusion comes from your loose interchange of "image" and "container". I think I know enough to parse those out without trouble now. But, for clarity, I don't think it makes sense to say anything is running in an image. I think if we were to be precise, then things run inside containers (as opposed to images), and containers are launched/created from an image. I think of images as just snapshots of a particular container's (stopped) state, from which another container can be launched. The build stage, therefore, just seems to be a starting up a container with one base image as input, and a new (possibly incremental CoW) image as output.
   
   > The final image layer generated is the last "layer" of changes that comprise the image and is tagged as the final image.
   > 
   > So it's different from building a virtual machine image where you only get an single output of a image based on all the commands run within the image at build.
   > 
   > This allows Docker to share image layers between containers and save space.
   
   Yeah, that just seems to be a basic copy-on-write incremental filesystem implementation. Not fundamentally different than qcow files and libvirtd... just a lot of easier-to-use tooling around those same fundamental concepts (and the primary fact that isolation comes from OS containerization features instead of virtualization for machine emulation). You still have to have a running container in which the build is executed. The docker documentation at https://docs.docker.com/engine/reference/commandline/build/ specifically refers to "the containers used in the build" and there are flags you can put on the command line to customize the cgroups for this running build container. So, there is actually a container running in the build stage, according to the docs... just not the same container you make later from the image that is the output of the build stage.
   
   
   > The github action only referenced publishing the contents of `_site` so I don't think the possible use of `/tmp` matters since it was never published before.
   
   Yeah, it shouldn't matter for anything we're doing... I'm just not sure how Docker might handle that... whether it would preserve those out-of-tree temporary file states or not.
   
   > So WORKDIR sets the PWD for any following RUN, COPY, ENTRYPOINT, and CMD commands in the Dockerfile.
   
   This is what I was noticing. Specifically, we're relying on it being the same for RUN and CMD, even though those run at completely separate phases. I was thinking that we might want to be in one directory for building the image, and another directory when we run the container. Not sure if there's a way to control that with that much granularity. Can always `(cd /somewhere else && dosomething)`, I suppose.
   
   > The project does support [adjusting for a baseurl](https://github.com/gjtorikian/html-proofer#adjusting-for-a-baseurl)
   
   :+1: 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178154227


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.

Review Comment:
   Comment added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii merged pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii merged PR #384:
URL: https://github.com/apache/accumulo-website/pull/384


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175922848


##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup

Review Comment:
   > I'm not super familiar with the base ruby image, but in my environment, the system gems are provided by system RPMs in my distribution. This has been my experience using CentOS/RHEL/Fedora. So, a `yum` (or `dnf`) update is normally sufficient and I don't need to do a `gem update`. Is the situation different for the base ruby image?
   > 
   The ruby image uses RubyGems for dependency management. In the Dockerfile we run the `apt-get update` command first to ensure that system packages are updated and also install some basic required packages. 
   
   `gem update --system` ensures that any Gems loaded from RubyGems are also updated before we attempt to install our application specific gems via `bundle install`. 
   
   Your comment got me curious so I popped into the image and verified there are no ruby-related packages installed via `apt`.  
   
   > Also, the instructions on the README in https://github.com/apache/accumulo-website#local-builds-for-testing explain in detail how to set up the GEM environment as a non-supervisor user for local gems installed to the user directory by bundler. I think it would be helpful if this Dockerfile did something similar, just automated, so instead of setting things up differently by doing system installed gems. That way, there's only a single set of instructions to follow, whether we're doing something via a local build, or trying to maintain this Dockerfile in future. However, it looks like this is relying on a different set of environmental assumptions from the those in the README. It looks like this is installing gems using the superuser, and gems are being installed to a system directory rather than a local user directory. I'm not 100% certain of this, since I'm not super familiar with Docker. If this is diverging from the detailed README instructions, it would be helpful to have som
 e inline comments explaining what's being done here, so we have a better chance of being able to maintain it going forward.
   
   Docker is operating on it's own separate filesystem layer so nothing installed in the container will affect anything on the given system. This accomplishes the same separation as setting `GEM_HOME` to a user directory. 
   
   If you're interested, the [Ruby image spec](https://hub.docker.com/_/ruby) calls out various environment variables that are set in the image, including `GEM_HOME` which is set to `/usr/local/bundle`. 
   
   The explicit `gem install bundler` command isn't needed as the Gemfile.lock requires Ruby 2.7 to be the 
   minimum installed and bundler was added as a default gem in 2.6. 
   
   So, I don't see this diverging from the detailed readme instructions as `GEM_HOME` is already set in the container and the next command to run is `bundle install`. 
   
   As long as all of the required dependencies are defined in the Gemfile, then we can ensure that all development environments are held to the same standard. 
   
   The **only** action that would require an update to the Dockerfile is a Ruby version update. But that is still extremely simple as you would just update the `FROM` image tag and run the documented `_/scripts/build-images.sh` script .
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] dlmarion commented on a diff in pull request #384: Add containerized development environment

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1169926268


##########
README.md:
##########
@@ -108,6 +108,31 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Docker dev environment 

Review Comment:
   ```suggestion
   ### Testing using Docker environment
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] dlmarion commented on pull request #384: Add containerized development environment

Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1516459507

   I would use the Docker container to test changes to the website locally. Like I said, I have had issues getting everything that is currently required installed. Using a Docker image that provides a working environment for everyone, without everyone having to go to the trouble of finding and installing all of the dependencies, sounds great.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1515711160

   > I wouldn't discount the use of a containerized environment for doing website changes as I've had conversations with at least two accumulo devs who plan on using it besides myself.
   
   I'm not discounting it. I'm fine with it. I'm just saying I'm uncertain about how much use it will get, given that none of the people who have been doing the work on the website have had a problem updating the website in its absence.
   
   > Overall I've found that build environments that can be quickly rebuilt and shared helps reduce "works on my machine" issues and update rot.
   
   I agree, but I don't find Docker any quicker than GitHub Actions builds or local Jekyll bundler builds without Docker. But, it doesn't matter, because I am fine with adding the Dockerfile, even if it gets no use at all.
   
   > The gem lock updates were mostly generated from the containerized version, however I'm happy to spend more time seeing if I can eliminate those differences.
   
   I think the issues I saw were related to the fact that I have a *much* newer version of GCC than the native extensions of those gems were developed with. Normally, I'd be fine with updating Jekyll, and then updating any dependent gems. But, if those gems' native extensions can't be built with the latest GCC, then that's going to cause bugs temporarily until those issues are fixed. This is just a timing issue... we're currently trying to make some updates to the site, and those breaking changes are coming at a bad time. There is nothing wrong with the previous gem versions, so far as I know. They might have some vulnerabilities or bugs or features that we're not encountering, but that's not causing a problem in our isolated build environments with known input, so until we can figure that out, I think the gems should just stay locked where they are.
   
   > I explicitly did not add the htmlproofer to the GitHub build step as it currently finds thousands of html linking errors in the current site.
   > 
   > A large majority of these findings are valid and I did not want to add a failing build step that would block other documentation work.
   
   That's why I think it should serve as its own PR, after those links are fixed, or in conjunction with those fixes, rather than included as part of a completely unrelated Dockerfile PR.
   
   For what it's worth, I have previously used a Python-based Link Checker to manually check stuff using a local `bundle exec jekyll serve`. That might be easier to configure in GitHub Actions... I'm not sure. I just know the crawler/checker itself does not necessarily need to be ruby-based just because the static site is built with ruby. I don't have a preference.
   
   The main thing regarding the link checking is that it's just wholly unrelated to the Docker stuff, and can easily be its own change, along with the actual fixes to the broken links. It's a good QA thing to add... it's just not related or dependent on the local build using Docker. It would be related, however, to the fixes of any broken links.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#issuecomment-1520832513

   @ctubbsii I worked on this a bit more and I was able to add a docker environment without modifying the original Gemfile or Gemfile.lock. 
   
   This PR adds two containers, one is the containerized build environment.
   The other is a validation container that can be run against any directory that contains rendered html. 
   
   This should allow your local development environment to remain unchanged while still providing a containerized solution. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175987200


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.
+
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.

Review Comment:
   Given the verbosity of `--mount`, and the ubiquity of `-v`, I'm inclined to agree `-v` is better. But, I think the wording in the README should make it more clear what's happening. Instead of saying `Mounting... is recommended`, which sounds like we're recommending the user *do something extra*... we say something like:
   
   ```
   The `-v` flag mounts the directory to ensure that updates to the directory inside
   the container are reflected in your local environment so you can create a commit
   and submit a PR.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ddanielr commented on a diff in pull request #384: Add containerized development environment

Posted by "ddanielr (via GitHub)" <gi...@apache.org>.
ddanielr commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1175970787


##########
Dockerfile:
##########
@@ -0,0 +1,30 @@
+FROM ruby:2.7.8-slim-bullseye as base
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    curl \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /site
+
+COPY Gemfile /site/Gemfile
+COPY Gemfile.lock /site/Gemfile.lock
+
+RUN gem update --system && bundle install && gem cleanup

Review Comment:
   Sounds good! I'll put that comment in tomorrow and get this closed up. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1177177643


##########
README.md:
##########
@@ -108,6 +108,73 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+./_scripts/build-images.sh
+```
+
+This action will produce two containers: `webdev` and `webdev-validator`.
+The webdev container will execute a `jekyll serve` command with the
+polling option enabled.
+
+This provides the ability to immediately review rendered content changes.
+
+```bash
+docker run -d -v "$PWD":/site -p 4000:4000 webdev
+```
+
+Shell access can be obtained by overriding the default container command.
+
+This is useful for adding new gems, or modifying the Gemfile.lock for updating
+existing dependencies.
+
+```bash
+docker run -v "$PWD":/site -it webdev /bin/bash
+```
+
+Mounting the local directory as a volume is recommended to ensure that Gemfile and
+Gemfile.lock stay updated with any dependency changes.

Review Comment:
   Now that I'm understanding the Docker stuff better, I think we can streamline the container builds by just having one container (see earlier comment). I'll re-review the doc updates after that stuff is resolved. I actually still think the validation stuff should be done in a separate PR entirely. That would streamline this PR and we could get this merged in right away, doc updates included. With the validator stuff still up in the air (two containers vs. one container), I'm not sure what changes might still need to be made in the docs.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [accumulo-website] ctubbsii commented on a diff in pull request #384: Add containerized development environment

Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on code in PR #384:
URL: https://github.com/apache/accumulo-website/pull/384#discussion_r1178133523


##########
README.md:
##########
@@ -108,6 +108,50 @@ HTML styled "just right".
 Jekyll will print a local URL where the site can be viewed (usually,
 [http://0.0.0.0:4000/](http://0.0.0.0:4000/)).
 
+### Testing using Docker environment 
+
+#### Build environment
+A containerized development environment can be built using the local
+Dockerfile.
+
+Run the build-images.sh script to generate the development environment and
+associated images.
+
+```bash
+docker build -t webdev .
+```
+
+This action will produce a `webdev` container which will execute a
+`jekyll serve` command with the polling option enabled.
+
+The webdev container does not store any site content inside of it.
+For rendering to function, the local directory must be mounted inside
+the container in a read/write configuration.
+
+This mounting is accomplished with the volume `-v` flag in the
+following `docker run` command.

Review Comment:
   ```suggestion
   When you run a container using the webdev image, your current working
   directory will be mounted, so that any changes made by the build inside
   the container will be reflected in your local workspace. This is done with the
   `-v` flag. To run the container, execute the following command:
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org