You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by tv...@apache.org on 2023/06/07 20:20:31 UTC
[beam] branch master updated: Disable kaniko cache compression when prebuilding images (#27023)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 2a5ac112532 Disable kaniko cache compression when prebuilding images (#27023)
2a5ac112532 is described below
commit 2a5ac1125324dc4e79ce6b496400d0750f895cea
Author: David Cavazos <dc...@google.com>
AuthorDate: Wed Jun 7 13:20:20 2023 -0700
Disable kaniko cache compression when prebuilding images (#27023)
---
CHANGES.md | 1 +
.../apache_beam/runners/portability/sdk_container_builder.py | 8 +++++++-
2 files changed, 8 insertions(+), 1 deletion(-)
diff --git a/CHANGES.md b/CHANGES.md
index 6ab0852f2b2..5c7733950ae 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -62,6 +62,7 @@
## New Features / Improvements
+* Allow prebuilding large images when using `--prebuild_sdk_container_engine=cloud_build`, like images depending on `tensorflow` or `torch` ([#27023](https://github.com/apache/beam/pull/27023))
* X feature added (Java/Python) ([#X](https://github.com/apache/beam/issues/X)).
## Breaking Changes
diff --git a/sdks/python/apache_beam/runners/portability/sdk_container_builder.py b/sdks/python/apache_beam/runners/portability/sdk_container_builder.py
index f81e015ea59..19becd3e123 100644
--- a/sdks/python/apache_beam/runners/portability/sdk_container_builder.py
+++ b/sdks/python/apache_beam/runners/portability/sdk_container_builder.py
@@ -252,7 +252,13 @@ class _SdkContainerImageCloudBuilder(SdkContainerImageBuilder):
build.steps = []
step = cloudbuild.BuildStep()
step.name = 'gcr.io/kaniko-project/executor:latest'
- step.args = ['--destination=' + container_image_name, '--cache=true']
+ # Disable compression caching to allow for large images to be cached.
+ # See: https://github.com/GoogleContainerTools/kaniko/issues/1669
+ step.args = [
+ '--destination=' + container_image_name,
+ '--cache=true',
+ '--compressed-caching=false',
+ ]
step.dir = SOURCE_FOLDER
build.steps.append(step)