You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2021/07/02 07:36:10 UTC
[GitHub] [flink] zentol commented on a change in pull request #16332: [FLINK-22386] Introduce cache for docker images used during testing
zentol commented on a change in pull request #16332:
URL: https://github.com/apache/flink/pull/16332#discussion_r662803072
##########
File path: tools/azure-pipelines/jobs-template.yml
##########
@@ -160,6 +176,13 @@ jobs:
targetPath: $(DEBUG_FILES_OUTPUT_DIR)
artifact: logs-${{parameters.stage_name}}-$(DEBUG_FILES_NAME)
+ - script: |
+ mkdir -p $(TESTCONTAINER_CACHE_FOLDER)
+ docker save $(docker image ls --format "{{.Repository}}:{{.Tag}}") -o $(TESTCONTAINER_CACHE_FOLDER)/cache.tar || true
Review comment:
I'm wondering if we'd should filter some of these out. Even the [core stage](https://dev.azure.com/Autophagy/Flink-CI/_build/results?buildId=80&view=logs&j=9dc1b5dc-bcfa-5f83-eaa7-0cb181ddc267&t=95a75a7f-839b-5434-cae3-c0698e9852d8), which doesn't use test containers at all still caches 4-5gigs. (consuming 8 minutes in the process)
##########
File path: tools/azure-pipelines/build-apache-repo.yml
##########
@@ -47,6 +47,9 @@ variables:
MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
CACHE_KEY: maven | $(Agent.OS) | **/pom.xml, !**/target/**
CACHE_FALLBACK_KEY: maven | $(Agent.OS)
+ TESTCONTAINER_CACHE_KEY: testcontainer-cache | $(Agent.OS) | **/pom.xml, !**/target/**
Review comment:
```suggestion
TESTCONTAINER_CACHE_KEY: testcontainer-cache | $(Agent.OS)
```
The used images are usually unrelated to the poms, so I'm not sure whether using the poms as a key makes sense.
##########
File path: tools/azure-pipelines/jobs-template.yml
##########
@@ -129,6 +129,22 @@ jobs:
condition: not(eq('${{parameters.test_pool_definition.name}}', 'Default'))
displayName: Cache Maven local repo
+ - task: Cache@2
+ inputs:
+ key: '"$(module)" | $(TESTCONTAINER_CACHE_KEY)'
+ restoreKeys: '"$(module)" | $(TESTCONTAINER_CACHE_FALLBACK_KEY)'
+ path: $(TESTCONTAINER_CACHE_FOLDER)
+ cacheHitVar: TESTCONTAINER_CACHE_HIT
+ continueOnError: true
+ condition: not(eq('${{parameters.test_pool_definition.name}}', 'Default'))
+ displayName: Cache Testcontainer docker images
+
+ - script: |
+ docker load -i $(TESTCONTAINER_CACHE_FOLDER)/cache.tar || true
+ displayName: Restore Testcontainer docker images
+ condition: and(not(canceled()), eq(variables.TESTCONTAINER_CACHE_HIT, 'true'))
+ continueOnError: true
Review comment:
Based on [this build](https://dev.azure.com/Autophagy/Flink-CI/_build/results?buildId=80&view=logs&j=9dc1b5dc-bcfa-5f83-eaa7-0cb181ddc267&t=b1670b3c-3f19-5f34-3e91-67ac7d80384f) this doesn't seem to work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org