You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "MartijnVisser (via GitHub)" <gi...@apache.org> on 2026/09/14 11:46:14 UTC

[PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

MartijnVisser opened a new pull request, #29173:
URL: https://github.com/apache/flink/pull/29173

   ## What is the purpose of the change
   
   The `minio/minio` repository no longer exists on Docker Hub, so every test that starts the MinIO container fails on this branch:
   
   ```
   Caused by: com.github.dockerjava.api.exception.NotFoundException: Status 404: {"message":"pull access denied for minio/minio, repository does not exist or may require 'docker login'"}
   ```
   
   This points the image at quay.io, which still serves the same tag at the same digest.
   
   master is not affected because FLINK-39499 replaced MinIO with SeaweedFS there, so there is no master PR for this one. The same change goes to the other two affected release branches.
   
   ## Brief change log
   
     - `DockerImageVersions.MINIO` now points at `quay.io/minio/minio:RELEASE.2022-02-07T08-17-33Z`, which resolves to the same digest Docker Hub served (`sha256:7dda745aefd6152f0d04fdd23377f9e52549df3fc4307f16b8bc562ae2b8119f`)
     - `common_s3_minio.sh` pulled the image untagged and is now pinned to that same tag. Current MinIO images fail with `Unable to use the drive /data` on a plain bind mount, so the tag cannot be left floating
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as `MinioTestContainerTest`, the `HAJobRunOnMinioS3Store` and `S5CmdOnMinio` ITCases, and the s3 end-to-end tests.
   
   Verified locally on this branch:
   
     - Before the change, `MinioTestContainerTest` fails 6/6 with the `pull access denied` error above
     - After the change, `MinioTestContainerTest` passes 6/6
     - The pinned image was also started by hand with the same arguments `common_s3_minio.sh` uses (bind mount, `-P`, `MINIO_ACCESS_KEY`/`MINIO_SECRET_KEY`) and serves the S3 API
   
   The s3 end-to-end scripts were not run end to end locally, and `mvn clean verify` was not run in full; the affected module was built and tested directly.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
     - The S3 file system connector: no (test infrastructure only)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes (please specify the tool below)
   
   Generated-by: Claude Opus 5 (1M context)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser merged PR #29173:
URL: https://github.com/apache/flink/pull/29173


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "MartijnVisser (via GitHub)" <gi...@apache.org>.
MartijnVisser commented on PR #29173:
URL: https://github.com/apache/flink/pull/29173#issuecomment-5664697597

   @spuru9 the e2e script pulled latest and the last green nightly on 2026-09-11 got sha256:14cea493, which is RELEASE.2025-09-07T16-13-09Z. I have pinned that instead.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "spuru9 (via GitHub)" <gi...@apache.org>.
spuru9 commented on code in PR #29173:
URL: https://github.com/apache/flink/pull/29173#discussion_r4005340650


##########
flink-end-to-end-tests/test-scripts/common_s3_minio.sh:
##########
@@ -44,11 +44,12 @@ S3_TEST_DATA_WORDS_URI="s3://$IT_CASE_S3_BUCKET/words"
 ###################################
 function s3_start {
   echo "Spawning minio for s3 tests"
+  # Pinned: current MinIO images no longer serve a plain directory ("Unable to use the drive /data").
   export MINIO_CONTAINER_ID=$(docker run -d \
     -P \
     --mount type=bind,source="$TEST_INFRA_DIR",target=/data \
     -e "MINIO_ACCESS_KEY=$AWS_ACCESS_KEY_ID" -e "MINIO_SECRET_KEY=$AWS_SECRET_ACCESS_KEY" -e "MINIO_DOMAIN=localhost" \
-    minio/minio \
+    quay.io/minio/minio:RELEASE.2022-02-07T08-17-33Z \

Review Comment:
   copy nit: We can use the latest version available RELEASE.2025-09-07T16-13-09Z?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29173:
URL: https://github.com/apache/flink/pull/29173#issuecomment-5663477107

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "add953a09707b5cd98f713e7b3fad56fc547bb9d",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "add953a09707b5cd98f713e7b3fad56fc547bb9d",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * add953a09707b5cd98f713e7b3fad56fc547bb9d UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "spuru9 (via GitHub)" <gi...@apache.org>.
spuru9 commented on code in PR #29173:
URL: https://github.com/apache/flink/pull/29173#discussion_r4005340650


##########
flink-end-to-end-tests/test-scripts/common_s3_minio.sh:
##########
@@ -44,11 +44,12 @@ S3_TEST_DATA_WORDS_URI="s3://$IT_CASE_S3_BUCKET/words"
 ###################################
 function s3_start {
   echo "Spawning minio for s3 tests"
+  # Pinned: current MinIO images no longer serve a plain directory ("Unable to use the drive /data").
   export MINIO_CONTAINER_ID=$(docker run -d \
     -P \
     --mount type=bind,source="$TEST_INFRA_DIR",target=/data \
     -e "MINIO_ACCESS_KEY=$AWS_ACCESS_KEY_ID" -e "MINIO_SECRET_KEY=$AWS_SECRET_ACCESS_KEY" -e "MINIO_DOMAIN=localhost" \
-    minio/minio \
+    quay.io/minio/minio:RELEASE.2022-02-07T08-17-33Z \

Review Comment:
   Any reason we used this version when a later version `RELEASE.2025-09-07T16-13-09Z`is also available?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [FLINK-40652][tests] Pull MinIO from quay.io instead of the removed Docker Hub repository [flink]

Posted by "spuru9 (via GitHub)" <gi...@apache.org>.
spuru9 commented on code in PR #29173:
URL: https://github.com/apache/flink/pull/29173#discussion_r4005340650


##########
flink-end-to-end-tests/test-scripts/common_s3_minio.sh:
##########
@@ -44,11 +44,12 @@ S3_TEST_DATA_WORDS_URI="s3://$IT_CASE_S3_BUCKET/words"
 ###################################
 function s3_start {
   echo "Spawning minio for s3 tests"
+  # Pinned: current MinIO images no longer serve a plain directory ("Unable to use the drive /data").
   export MINIO_CONTAINER_ID=$(docker run -d \
     -P \
     --mount type=bind,source="$TEST_INFRA_DIR",target=/data \
     -e "MINIO_ACCESS_KEY=$AWS_ACCESS_KEY_ID" -e "MINIO_SECRET_KEY=$AWS_SECRET_ACCESS_KEY" -e "MINIO_DOMAIN=localhost" \
-    minio/minio \
+    quay.io/minio/minio:RELEASE.2022-02-07T08-17-33Z \

Review Comment:
   copy nit: We can use the latest version available RELEASE.2025-09-07T16-13-09Z?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org