[GitHub] [pulsar] yaalsn commented on a change in pull request #14819: [CI] Refactor GitHub Actions based Pulsar CI

[GitBox <gi...@apache.org>]

yaalsn commented on a change in pull request #14819:
URL: https://github.com/apache/pulsar/pull/14819#discussion_r833181215



##########
File path: build/pulsar_ci_tool.sh
##########
@@ -46,6 +47,123 @@ function ci_dependency_check() {
   _ci_mvn -Pmain,skip-all,skipDocker,owasp-dependency-check initialize verify -pl '!pulsar-client-tools-test' "$@"
 }
 
+# installs a tool executable if it's not found on the PATH
+function ci_install_tool() {
+  local tool_executable=$1
+  local tool_package=${2:-$1}
+  if ! command -v $tool_executable &>/dev/null; then
+    echo "::group::Installing ${tool_package}"
+    sudo apt-get -y install ${tool_package} >/dev/null
+    echo '::endgroup::'
+  fi
+}
+
+# outputs the given message to stderr and exits the shell script
+function fail() {
+  echo "$*" >&2
+  exit 1
+}
+
+# function to retry a given commend 3 times with a backoff of 10 seconds in between
+function ci_retry() {
+  local n=1
+  local max=3
+  local delay=10
+  while true; do
+    "$@" && break || {
+      if [[ $n -lt $max ]]; then
+        ((n++))
+        echo "::warning::Command failed. Attempt $n/$max:"
+        sleep $delay
+      else
+        fail "::error::The command has failed after $n attempts."
+      fi
+    }
+  done
+}
+
+# saves a given image (1st parameter) to the GitHub Actions cache with the given key (2nd parameter)
+function ci_docker_save_image_to_github_actions_cache() {
+  local image=$1
+  local cachekey=$2
+  ci_install_tool pv
+  echo "::group::Saving docker image ${image} with key ${cachekey} in GitHub Actions Cache"
+  ci_retry bash -c "docker save ${image} | zstd | pv -ft -i 5 | pv -Wbaf -i 5 | curl -s -H 'Content-Type: application/octet-stream' -X PUT --data-binary @- http://localhost:12321/${cachekey}"
+  echo "::endgroup::"
+}
+
+# loads a docker image from the GitHub Actions cache with the given key (1st parameter)
+function ci_docker_load_image_from_github_actions_cache() {
+  local cachekey=$1
+  ci_install_tool pv
+  echo "::group::Loading docker image from key ${cachekey} in GitHub Actions Cache"
+  ci_retry bash -c "curl -s http://localhost:12321/${cachekey} | pv -batf -i 5 | unzstd | docker load"

Review comment:
       @lhotari I am sorry but I still want to confirm with you that if this `http-cache-action` uses `github action cache`, just like this one https://github.com/actions/cache. And It seems that every repository has a 10GB cache limit total not every PR.
   <img width="990" alt="image" src="https://user-images.githubusercontent.com/10069311/159694970-a8f7f79e-5a93-4576-8a57-225933416d1a.png">
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org