You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/09/08 18:19:15 UTC
[GitHub] [druid] FrankChen021 opened a new pull request, #13059: Some improvements about Docker
FrankChen021 opened a new pull request, #13059:
URL: https://github.com/apache/druid/pull/13059
These some improvements about Docker, including:
1. Update to run with JRE 11 by default
2. Update to use [gcr.io/distroless/java11-debian11](https://github.com/GoogleContainerTools/distroless) image as base by default. This image is currently supported image for Java 11.
3. Enable docker buildkit cache to speed up building
4. Download [bash-static](https://github.com/robxu9/bash-static) to the image so that scripts that require bash can be executed(Fixes #13057 )
5. bump builder image from `3.8.4-jdk-11-slim` to the latest `3.8.6-jdk-11-slim`
This PR has:
- [X] been self-reviewed.
- [ ] using the [concurrency checklist](https://github.com/apache/druid/blob/master/dev/code-review/concurrency.md) (Remove this item if the PR doesn't have any relation to concurrency.)
- [ ] added documentation for new or modified features or behaviors.
- [ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
- [ ] added or updated version, license, or notice information in [licenses.yaml](https://github.com/apache/druid/blob/master/dev/license.md)
- [ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
- [ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met.
- [ ] added integration tests.
- [X] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 merged pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 merged PR #13059:
URL: https://github.com/apache/druid/pull/13059
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969819083
##########
distribution/docker/Dockerfile:
##########
@@ -17,8 +17,10 @@
# under the License.
#
-ARG JDK_VERSION=8
-FROM maven:3.8.4-jdk-11-slim as builder
+ARG JDK_VERSION=11
+
+FROM maven:3.8.6-jdk-11-slim as builder
Review Comment:
It's a very good idea.
But for Mac M1/M2, as I said above, it takes too long to build the distribution file at the build stage. I will also add some instructions in the doc to guide people to build the distribution out of docker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967250873
##########
distribution/docker/README.md:
##########
@@ -19,22 +19,29 @@
## Build
-From the root of the repo, run `docker build -t apache/druid:tag -f distribution/docker/Dockerfile .`
+From the root of the repo, run following command:
-## Run
-
-Edit `environment` to suite. Run `docker-compose -f distribution/docker/docker-compose.yml up`
+```bash
+DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .
Review Comment:
I was wondering if the need for buildkit would impact our Docker Hub autobuilds, but it looks like it's already enable by default on docker hub https://docs.docker.com/docker-hub/builds/#build-images-with-buildkit
(we still need it in the docs of course)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969814051
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
@xvrl Let's download the bash-static based on the target arch in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r968946187
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
> we still can't build the image on Mac M1
We can tackle building on arm and providing arm docker images separately. It should be possible to cross-compile the console until the node.js issue is resolved. That means building on M1 would be slower due to emulation, but I think that's better than not building at all.
> Current Dockerfile uses amd64 busybox
it looks like you solved that problem by pointing to a multi-arch build of busybox.
I think the only issue left to solve is whether we want to switch to debian-slim, or if we want to add the logic to download the right arch binary for bash-static. The latter one might be simpler for now and we can do that by adding an if statement checking the value of `$TARGETARCH` (see https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope) to pick the right bash-static binary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969073895
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
@gianm What's your opinion on changing the base image from distroless to debian-11?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r968946187
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
> we still can't build the image on Mac M1
We can tackle building on arm and providing arm docker images separately. It should be possible to cross-compile the console until the node.js issue is resolved. That means building on M1 would be slower due to emulation, but I think that's better than not building at all.
> Current Dockerfile uses amd64 busybox
it looks like you solved that problem by pointing to a multi-arch build of busybox.
I think the only issue left to solve is whether we want to switch to debian-slim, or if we want to add the logic to download the right arch binary for bash-static. The latter one might be simpler for now and we can do that by adding an if statement checking the value of `$TARGETARCH` (see https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope) to pick the right bash-static binary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on PR #13059:
URL: https://github.com/apache/druid/pull/13059#issuecomment-1248400331
@FrankChen021 you can go ahead and merge this, I don't think there is anything controversial about it. Adding bash-static is already backported as part of #13063. The biggest change is the switch to JDK 11, so we should probably mention that fact in the PR description and also make sure we add it to the release notes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967245125
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
how much work would it be to fix the script to not use bash specific constructs?
If we are going to install bash or other things, maybe it would make sense to switch to a openjdk debian-slim image that already includes it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967292756
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
how do others feel about switching base images? cc @gianm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969076109
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
@xvrl Actually I tried to build image on my Mac M1 by adding `--platform=linux/amd64` to the docker build command several days ago, and I didn't make it. It takes more than 20 minutes and it seems it's stuck at building the source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967292756
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
how do others feel about switching base images? I agree installing bash is not multi-arch friendly, plus we do run tests against ARM64 today, so it would be nice to also do that for docker. cc @gianm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on PR #13059:
URL: https://github.com/apache/druid/pull/13059#issuecomment-1242150533
After switching to JRE 11, we need to use `run-java` in the `druid.sh` to start Druid services.
Making as draft temporarily till this change is done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969978537
##########
distribution/docker/Dockerfile:
##########
@@ -29,28 +35,47 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
-FROM amd64/busybox:1.30.0-glibc as busybox
+FROM busybox:1.35.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+# Predefine builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
+ARG TARGETARCH
+
+#
+# Download bash-static binary to execute scripts that require bash.
+# Although bash-static supports multiple platforms, but there's no need for us to support all those platform, amd64 and arm64 are enough.
+#
+ARG BASH_URL_BASE="https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3"
+RUN if [ "$TARGETARCH" = "arm64" ]; then \
+ export BASH_URL=$(echo ${BASH_URL_BASE}/bash-linux-aarch64) ; \
Review Comment:
a few nits:
- no need to call `export` since we are not using the URL outside of this single command line.
- you also don't need to call `echo` and can interpolate directly
```suggestion
BASH_URL="${BASH_URL_BASE}/bash-linux-aarch64" ; \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r968950142
##########
distribution/docker/Dockerfile:
##########
@@ -17,8 +17,10 @@
# under the License.
#
-ARG JDK_VERSION=8
-FROM maven:3.8.4-jdk-11-slim as builder
+ARG JDK_VERSION=11
+
+FROM maven:3.8.6-jdk-11-slim as builder
Review Comment:
to make the build work on arm64 we could address the issue by forcing the platform to use amd64 using `FROM --platform=linux/amd64` (let's make sure we add a comment why and link to the node issue)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967251438
##########
distribution/docker/README.md:
##########
@@ -19,22 +19,29 @@
## Build
-From the root of the repo, run `docker build -t apache/druid:tag -f distribution/docker/Dockerfile .`
+From the root of the repo, run following command:
-## Run
-
-Edit `environment` to suite. Run `docker-compose -f distribution/docker/docker-compose.yml up`
+```bash
+DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .
Review Comment:
do we need to enable buildkit when we build our integration test images?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on PR #13059:
URL: https://github.com/apache/druid/pull/13059#issuecomment-1246157779
I have built and run the image on my intel MacBook and M1 MacBook, both are OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969979435
##########
distribution/docker/Dockerfile:
##########
@@ -29,28 +35,47 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
-FROM amd64/busybox:1.30.0-glibc as busybox
+FROM busybox:1.35.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+# Predefine builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
+ARG TARGETARCH
+
+#
+# Download bash-static binary to execute scripts that require bash.
+# Although bash-static supports multiple platforms, but there's no need for us to support all those platform, amd64 and arm64 are enough.
+#
+ARG BASH_URL_BASE="https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3"
+RUN if [ "$TARGETARCH" = "arm64" ]; then \
+ export BASH_URL=$(echo ${BASH_URL_BASE}/bash-linux-aarch64) ; \
+ elif [ "$TARGETARCH" = "amd64" ]; then \
+ export BASH_URL=$(echo ${BASH_URL_BASE}/bash-linux-x86_64) ; \
+ else \
+ echo "Unknown archtecture($TARGETARCH)" && exit ; \
Review Comment:
we should call exit with a non-zero error code to ensure the build fails when building with an unsupported architecture
```suggestion
echo "Unknown archtecture($TARGETARCH)" && exit 1; \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r971278151
##########
distribution/docker/Dockerfile:
##########
@@ -29,28 +35,47 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
-FROM amd64/busybox:1.30.0-glibc as busybox
+FROM busybox:1.35.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+# Predefined builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
+ARG TARGETARCH
+
+#
+# Download bash-static binary to execute scripts that require bash.
+# Although bash-static supports multiple platforms, but there's no need for us to support all those platform, amd64 and arm64 are enough.
+#
+ARG BASH_URL_BASE="https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3"
+RUN if [ "$TARGETARCH" = "arm64" ]; then \
+ BASH_URL="${BASH_URL_BASE}/bash-linux-aarch64" ; \
+ elif [ "$TARGETARCH" = "amd64" ]; then \
+ BASH_URL="${BASH_URL_BASE}/bash-linux-x86_64" ; \
+ else \
+ echo "Unsupported archtecture ($TARGETARCH)" && exit 1; \
Review Comment:
nit typo
```suggestion
echo "Unsupported architecture ($TARGETARCH)" && exit 1; \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967277293
##########
distribution/docker/README.md:
##########
@@ -19,22 +19,29 @@
## Build
-From the root of the repo, run `docker build -t apache/druid:tag -f distribution/docker/Dockerfile .`
+From the root of the repo, run following command:
-## Run
-
-Edit `environment` to suite. Run `docker-compose -f distribution/docker/docker-compose.yml up`
+```bash
+DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .
Review Comment:
we do for our k8s integration tests, where we use the default image.https://github.com/apache/druid/blob/master/integration-tests/script/setup_druid_on_k8s.sh#L35-L36
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on PR #13059:
URL: https://github.com/apache/druid/pull/13059#issuecomment-1248810725
Thank you @xvrl very much for you helpful review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967261565
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
I also agree with you that we switch to another base image. There're two reasons:
1. Installing bash in this way makes it hard to support ARM64 in the future
2. gcr.io/distroless image can't be directly accessed in mainland Chain due to network restriction, developers in China has to use some proxy servers to download this base image.
But distroless has some advantages such as smaller size, more security.
So currently I don't have an opinion on which we should use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967270505
##########
distribution/docker/README.md:
##########
@@ -19,22 +19,29 @@
## Build
-From the root of the repo, run `docker build -t apache/druid:tag -f distribution/docker/Dockerfile .`
+From the root of the repo, run following command:
-## Run
-
-Edit `environment` to suite. Run `docker-compose -f distribution/docker/docker-compose.yml up`
+```bash
+DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .
Review Comment:
Our it test images are not based on the distribution docker file, so I don't touch this part in this PR.
We can enable it and wait to see if the travis works fine well with buildkit in a follow up PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] xvrl commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
xvrl commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r969979435
##########
distribution/docker/Dockerfile:
##########
@@ -29,28 +35,47 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
-FROM amd64/busybox:1.30.0-glibc as busybox
+FROM busybox:1.35.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+# Predefine builtin arg, see: https://docs.docker.com/engine/reference/builder/#automatic-platform-args-in-the-global-scope
+ARG TARGETARCH
+
+#
+# Download bash-static binary to execute scripts that require bash.
+# Although bash-static supports multiple platforms, but there's no need for us to support all those platform, amd64 and arm64 are enough.
+#
+ARG BASH_URL_BASE="https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3"
+RUN if [ "$TARGETARCH" = "arm64" ]; then \
+ export BASH_URL=$(echo ${BASH_URL_BASE}/bash-linux-aarch64) ; \
+ elif [ "$TARGETARCH" = "amd64" ]; then \
+ export BASH_URL=$(echo ${BASH_URL_BASE}/bash-linux-x86_64) ; \
+ else \
+ echo "Unknown archtecture($TARGETARCH)" && exit ; \
Review Comment:
we should call exit with a non-zero error code to ensure the build fails when building with an unsupported architecture
```suggestion
echo "Unknown archtecture ($TARGETARCH)" && exit 1; \
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on PR #13059:
URL: https://github.com/apache/druid/pull/13059#issuecomment-1247623717
@gianm Do you have any comments on these changes? If no, I will merge it once the CI finishes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967281850
##########
distribution/docker/README.md:
##########
@@ -19,22 +19,29 @@
## Build
-From the root of the repo, run `docker build -t apache/druid:tag -f distribution/docker/Dockerfile .`
+From the root of the repo, run following command:
-## Run
-
-Edit `environment` to suite. Run `docker-compose -f distribution/docker/docker-compose.yml up`
+```bash
+DOCKER_BUILDKIT=1 docker build -t apache/druid:tag -f distribution/docker/Dockerfile .
Review Comment:
Let me update the command in this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] FrankChen021 commented on a diff in pull request #13059: Some improvements about Docker
Posted by GitBox <gi...@apache.org>.
FrankChen021 commented on code in PR #13059:
URL: https://github.com/apache/druid/pull/13059#discussion_r967573984
##########
distribution/docker/Dockerfile:
##########
@@ -29,27 +31,29 @@ RUN export DEBIAN_FRONTEND=noninteractive \
COPY . /src
WORKDIR /src
-RUN if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
+RUN --mount=type=cache,target=/root/.m2 if [ "$BUILD_FROM_SOURCE" = "true" ]; then \
mvn -B -ff -q dependency:go-offline \
install \
-Pdist,bundle-contrib-exts \
-Pskip-static-checks,skip-tests \
-Dmaven.javadoc.skip=true \
; fi
-RUN VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
+RUN --mount=type=cache,target=/root/.m2 VERSION=$(mvn -B -q org.apache.maven.plugins:maven-help-plugin:3.2.0:evaluate \
-Dexpression=project.version -DforceStdout=true \
) \
&& tar -zxf ./distribution/target/apache-druid-${VERSION}-bin.tar.gz -C /opt \
&& mv /opt/apache-druid-${VERSION} /opt/druid
FROM amd64/busybox:1.30.0-glibc as busybox
-FROM gcr.io/distroless/java:$JDK_VERSION
+FROM gcr.io/distroless/java$JDK_VERSION-debian11
LABEL maintainer="Apache Druid Developers <de...@druid.apache.org>"
COPY --from=busybox /bin/busybox /busybox/busybox
RUN ["/busybox/busybox", "--install", "/bin"]
+RUN wget https://github.com/robxu9/bash-static/releases/download/5.1.016-1.2.3/bash-linux-x86_64 -O /bin/bash \
+ && chmod 755 /bin/bash
Review Comment:
It's better to have ARM64 supported. There are some feedback that ask for ARM64 images(see #11820), and someone made it by themselves.
To support ARM64, we need be able to build and run on ARM64 based machines.
Current Dockerfile uses amd64 busybox which blocks build image to run on ARM64 machines. It's easy to solve.
Another problem is even we solve that, we still can't build the image on Mac M1. Current the web-console dependency still prevents us from doing this, see #13012
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org