You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@submarine.apache.org by by...@apache.org on 2021/06/29 09:17:37 UTC
[submarine] branch master updated: SUBMARINE-857. Image should be
built locally in github-action
This is an automated email from the ASF dual-hosted git repository.
byronhsu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/submarine.git
The following commit(s) were added to refs/heads/master by this push:
new 1c77153 SUBMARINE-857. Image should be built locally in github-action
1c77153 is described below
commit 1c77153504a0f4ebe9daef68751db657bb9bea9e
Author: ByronHsu <by...@gmail.com>
AuthorDate: Mon Jun 28 17:36:06 2021 +0800
SUBMARINE-857. Image should be built locally in github-action
### What is this PR for?
The current image is pulled from docker hub so the change in images will not be reflected immediately.
I revised the workflow files and build the images before deploying the cluster.
Also, I used Github action cache to cache docker layer to accelerate.
### What type of PR is it?
[Bug Fix]
### Todos
### What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-867
### How should this be tested?
### Screenshots (if appropriate)
### Questions:
* Do the license files need updating? No
* Are there breaking changes for older versions? No
* Does this need new documentation? No
Author: ByronHsu <by...@gmail.com>
Author: jeff-901 <b0...@ntu.edu.com>
Signed-off-by: byronhsu <by...@apache.org>
Closes #613 from ByronHsu/build-image-locally and squashes the following commits:
fd258588 [ByronHsu] fix path
bbe3e861 [ByronHsu] debug maven build
0fe62e15 [ByronHsu] debug maven build
cf5f655f [ByronHsu] debug maven build
76d1b7ab [ByronHsu] debug maven build
9196726a [ByronHsu] debug maven build
eb32872c [ByronHsu] debug maven build
8fcd7e88 [ByronHsu] debug maven build
8fc3f6f0 [ByronHsu] debug maven build
fb9d6a6f [ByronHsu] monitor error
216f208c [ByronHsu] monitor error
b37e13c1 [ByronHsu] fix typo
96d4d816 [ByronHsu] try to use docker cache
ac60687e [ByronHsu] try to use docker cache
4dc1145f [ByronHsu] build before deploy
5ba38849 [ByronHsu] ensure the script is ran
35f1e945 [jeff-901] for travis
e0293dea [jeff-901] for travis
eb0adac5 [jeff-901] edit remain url
85650f0c [jeff-901] edit mlflow dockerfile s3 url
ed1cfe28 [jeff-901] delete fix ip
---
.github/scripts/build-image-locally.sh | 30 +++++++++++++++++++
.github/workflows/python.yml | 53 ++++++++++++++++++++++++++++++++--
2 files changed, 81 insertions(+), 2 deletions(-)
diff --git a/.github/scripts/build-image-locally.sh b/.github/scripts/build-image-locally.sh
new file mode 100755
index 0000000..d935f53
--- /dev/null
+++ b/.github/scripts/build-image-locally.sh
@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+SUBMARINE_VERSION="0.6.0-SNAPSHOT"
+FOLDER_LIST=("database" "mlflow" "submarine")
+IMAGE_LIST=("apache/submarine:database-${SUBMARINE_VERSION}" "apache/submarine:mlflow-${SUBMARINE_VERSION}" "apache/submarine:server-${SUBMARINE_VERSION}")
+
+for i in "${!IMAGE_LIST[@]}"
+do
+ echo "Build Image ${IMAGE_LIST[i]}"
+ echo "Execute ./dev-support/docker-images/${FOLDER_LIST[i]}/build.sh"
+ ./dev-support/docker-images/"${FOLDER_LIST[i]}"/build.sh
+ kind load docker-image "${IMAGE_LIST[i]}"
+done
+
diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml
index 283a5c6..5defb8e 100644
--- a/.github/workflows/python.yml
+++ b/.github/workflows/python.yml
@@ -15,7 +15,8 @@ jobs:
env:
PYTHON: ${{ matrix.python }}
steps:
- - uses: actions/checkout@v1
+ - name: Checkout project
+ uses: actions/checkout@v1
- name: Setup python environment
uses: actions/setup-python@v1
with:
@@ -38,7 +39,20 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- - uses: actions/checkout@v1
+ - uses: actions/checkout@v2
+ - name: Set up JDK 1.8
+ uses: actions/setup-java@v1
+ with:
+ java-version: '1.8'
+ - name: Set up Maven 3.6.3
+ uses: stCarolas/setup-maven@v4
+ with:
+ maven-version: 3.6.3
+ - name: Check version
+ run: |
+ mvn --version
+ java -version
+
- name: Create the kind config
run: |
cat <<EOF > ./kind-config-kind.yaml
@@ -73,6 +87,41 @@ jobs:
kubectl get pods -n kube-system
echo "current-context:" $(kubectl config current-context)
echo "environment-kubeconfig:" ${KUBECONFIG}
+
+ # Cache docker layer
+ - name: Cache docker layer
+ uses: satackey/action-docker-layer-caching@v0.0.11
+ # Ignore the failure of a step and avoid terminating the job.
+ continue-on-error: true
+
+ # Cache maven (for submarine server)
+ - uses: actions/cache@v1
+ with:
+ path: ~/.m2/repository/com
+ key: ${{ runner.os }}-maven-com-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-com-
+ - uses: actions/cache@v1
+ with:
+ path: ~/.m2/repository/org
+ key: ${{ runner.os }}-maven-org-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-org-
+ - uses: actions/cache@v1
+ with:
+ path: ~/.m2/repository/net
+ key: ${{ runner.os }}-maven-net-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-net-
+ - uses: actions/cache@v1
+ with:
+ path: ~/.m2/repository/io
+ key: ${{ runner.os }}-maven-io-${{ hashFiles('**/pom.xml') }}
+ restore-keys: |
+ ${{ runner.os }}-maven-io-
+ - name: Build Image locally
+ run: |
+ .github/scripts/build-image-locally.sh
- name: Start submarine
run: |
# Fix submarine-database start failed in kind. https://github.com/kubernetes/minikube/issues/7906
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@submarine.apache.org
For additional commands, e-mail: dev-help@submarine.apache.org