You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/04 13:33:19 UTC
[GitHub] [airflow] potiuk opened a new pull request #16797: Move ci images to ghcr.io
potiuk opened a new pull request #16797:
URL: https://github.com/apache/airflow/pull/16797
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16797:
URL: https://github.com/apache/airflow/pull/16797#issuecomment-877666838
All fixed, but it depends on #16780 which must be merged first
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16797:
URL: https://github.com/apache/airflow/pull/16797#issuecomment-878381040
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #16797:
URL: https://github.com/apache/airflow/pull/16797#issuecomment-879521535
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16797:
URL: https://github.com/apache/airflow/pull/16797#issuecomment-879608396
> So much duplication but I assume it's unavoidable.
We will remove some of those when we change breeze to pyhon. But yeah, some duplication when it comes to names of files at least is unavoidable I think as we have things out- and in- container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #16797:
URL: https://github.com/apache/airflow/pull/16797
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #16797: Move CI-integration images to ghcr.io
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #16797:
URL: https://github.com/apache/airflow/pull/16797#discussion_r663565668
##########
File path: CI.rst
##########
@@ -77,16 +77,10 @@ this image can be built only once and used by all the jobs running tests. The im
``COMMIT_SHA`` of the incoming Pull Request and the tests run in the Pull Request can simply pull such image
rather than build it from the scratch. Pulling such image takes ~ 1 minute, thanks to that we are saving
a lot of precious time for jobs.
-4
-We use `GitHub Container Registry <https://docs.github.com/en/packages/guides/about-github-container-registry>`_
-GitHub Package Registry ``GITHUB_TOKEN`` is needed to push to the registry. You also have to manually manage
-permissions of the images, after creating image for the first time (pushing it using your personal token)
-you need to set their visibility to ``Public`` and enable
-`Inheriting access from repository <https://docs.github.com/en/packages/learn-github-packages/configuring-a-packages-access-control-and-visibility#inheriting-access-for-a-container-image-from-a-repository>`_
-Those images have specific naming schema. See `Images documentation <IMAGES.rst>`_ for details.
-You can interact with the GitHub Registry images (pull/push) via `Breeze <BREEZE.rst>`_ - by passing
-``--use-github-registry`` flag.
+We use `GitHub Container Registry <https://docs.github.com/en/packages/guides/about-github-container-registry>`_
+``GITHUB_TOKEN`` is needed to push to the registry and we configured scopes of the tokens in our jobs
+to be able to write to the registry.
Review comment:
```suggestion
We use `GitHub Container Registry <https://docs.github.com/en/packages/guides/about-github-container-registry>`_.
``GITHUB_TOKEN`` is needed to push to the registry and we configured scopes of the tokens in our jobs
to be able to write to the registry.
```
##########
File path: BREEZE.rst
##########
@@ -1249,14 +1242,16 @@ This is the current syntax for `./breeze <./breeze>`_:
'--build-cache-local' or '-build-cache-pulled', or '--build-cache-none'
Choosing whether to force pull images or force build the image:
- '--force-build-image',
- '--force-pull-image', '--force-pull-base-python-image'
+ '--force-build-image', '--force-pull-image'
+
+ Checking if the base python image has been updated:
+ '--check-if-base-python-image-updated'
You can also pass '--production-image' flag to build production image rather than CI image.
- For DockerHub pulling of base images: '--dockerhub-user' and '--dockerhub-repo' flags can be
- used to specify the repository to pull from. For GitHub repository, the '--github-repository'
- flag can be used for the same purpose.
+ For GitHub repository, the '--github-repository' can be used to choose repository
+ to pull/push images.
Review comment:
```suggestion
For GitHub repository, '--github-repository' can be used to choose repository
to pull/push images.
```
##########
File path: BREEZE.rst
##########
@@ -1201,13 +1195,12 @@ This is the current syntax for `./breeze <./breeze>`_:
'breeze shell -- -c "ls -la"'
'breeze -- -c "ls -la"'
- For DockerHub pull: --dockerhub-user and --dockerhub-repo flags can be used to specify
- the repository to pull from. For GitHub repository, the --github-repository
- flag can be used for the same purpose. You can also use --github-image-id <COMMIT_SHA> in case
+ For GitHub repository, the --github-repository flag can be used to specify the repository
+ to pull and push images. You can also use --github-image-id <COMMIT_SHA> in case
you want to pull the image with specific COMMIT_SHA tag.
'breeze shell \
- --github-image-id 9a621eaa394c0a0a336f8e1b31b35eff4e4ee86e' - pull/use image with SHA
+ -- github-image-id 9a621eaa394c0a0a336f8e1b31b35eff4e4ee86e' - pull/use image with SHA
Review comment:
Typo?
##########
File path: CI.rst
##########
@@ -472,11 +411,10 @@ The following components are part of the CI infrastructure
* **GitHub Actions** - (GA) UI + execution engine for our jobs
* **GA CRON trigger** - GitHub Actions CRON triggering our jobs
* **GA Workers** - virtual machines running our jobs at GitHub Actions (max 20 in parallel)
-* **GitHub Private Image Registry**- image registry used as build cache for CI jobs.
- It is at https://ghcr.io/apache/airflow/airflow
-* **DockerHub Public Image Registry** - publicly available image registry at DockerHub.
- It is at https://hub.docker.com/r/apache/airflow-ci
-* **DockerHub Build Workers** - virtual machines running build jibs at DockerHub
+* **GitHub Image Registry**- image registry used as build cache for CI jobs.
+ It is at https://ghcr.io/apache/airflow
+* **DockerHub Image Registry**- image registry used to pull base Python images and (manually) publish
+ the released Production Airflow images. It is at https://dockerhub.com/apache/airflow
Review comment:
```suggestion
* **GitHub Image Registry** - image registry used as build cache for CI jobs.
It is at https://ghcr.io/apache/airflow
* **DockerHub Image Registry** - image registry used to pull base Python images and (manually) publish
the released Production Airflow images. It is at https://dockerhub.com/apache/airflow
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org