You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/03/08 14:44:48 UTC
[GitHub] [airflow] potiuk opened a new pull request #7656: [AIRFLOW-7013]
Automated check if Breeze image needs to be pulled
potiuk opened a new pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656
---
Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390568970
##########
File path: scripts/ci/_utils.sh
##########
@@ -552,6 +561,85 @@ function set_current_image_variables {
fi
}
+function build_image_manifest() {
+ verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+ verbose_docker build \
+ --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+ --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+ -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+ TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+ TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+ set +e
+ verbose_docker rm --force "local-airflow-manifest" >/dev/null 2>&1
+ if ! verbose_docker create --name "local-airflow-manifest" \
+ "${AIRFLOW_LOCAL_MANIFEST_IMAGE}" >/dev/null 2>&1 ; then
+ echo
+ echo "Local docker not available"
Review comment:
Added full description + a bit nicer output showing pull progress as well as step of the docker build during pre-commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7656: [AIRFLOW-7013]
Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
> Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.64%`.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7656 +/- ##
==========================================
- Coverage 86.98% 86.34% -0.65%
==========================================
Files 904 904
Lines 43736 43736
==========================================
- Hits 38045 37764 -281
- Misses 5691 5972 +281
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/api/auth/backend/kerberos\_auth.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvYXV0aC9iYWNrZW5kL2tlcmJlcm9zX2F1dGgucHk=) | `28.16% <0.00%> (-54.93%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
| [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0.00%> (-45.66%)` | :arrow_down: |
| [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
| [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390333381
##########
File path: scripts/ci/_utils.sh
##########
@@ -552,6 +561,85 @@ function set_current_image_variables {
fi
}
+function build_image_manifest() {
+ verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+ verbose_docker build \
+ --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+ --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+ -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+ TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+ TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+ set +e
+ verbose_docker rm --force "local-airflow-manifest" >/dev/null 2>&1
+ if ! verbose_docker create --name "local-airflow-manifest" \
+ "${AIRFLOW_LOCAL_MANIFEST_IMAGE}" >/dev/null 2>&1 ; then
+ echo
+ echo "Local docker not available"
Review comment:
```suggestion
echo "Local airflow CI docker image not available"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390460517
##########
File path: scripts/ci/_utils.sh
##########
@@ -552,6 +561,85 @@ function set_current_image_variables {
fi
}
+function build_image_manifest() {
+ verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+ verbose_docker build \
+ --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+ --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+ -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+ TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+ TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+ set +e
+ verbose_docker rm --force "local-airflow-manifest" >/dev/null 2>&1
+ if ! verbose_docker create --name "local-airflow-manifest" \
+ "${AIRFLOW_LOCAL_MANIFEST_IMAGE}" >/dev/null 2>&1 ; then
+ echo
+ echo "Local docker not available"
Review comment:
> Can you add some docs/comments in the PR somewhere about
>
> 1. what this is doing -- I'm having trouble following the logic; and
> 2. why (your comment in the PR should go in the script somewhere)
Good point. I added comments describing both.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
> Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7656 +/- ##
==========================================
- Coverage 86.98% 86.49% -0.49%
==========================================
Files 904 904
Lines 43736 43736
==========================================
- Hits 38045 37831 -214
- Misses 5691 5905 +214
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
| [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
| [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
| [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk merged pull request #7656: [AIRFLOW-7013]
Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated
check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597575192
Hey @ashb -> are you OK with those comments ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
> Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7656 +/- ##
==========================================
- Coverage 86.98% 86.49% -0.49%
==========================================
Files 904 904
Lines 43736 43736
==========================================
- Hits 38045 37831 -214
- Misses 5691 5905 +214
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
| [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
| [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
| [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated
check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-596213504
This one implements automated check if it is worth to pull image rather than build it locally. When the image is changed a lot (for example with buster) it makes very little sense to rebuild the image, it's better to pull it.
This change will automatically detect if it is worth to build or pull&build image when you reach the "need rebuild phase".
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390333670
##########
File path: scripts/ci/_utils.sh
##########
@@ -552,6 +561,85 @@ function set_current_image_variables {
fi
}
+function build_image_manifest() {
+ verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+ verbose_docker build \
+ --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+ --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+ -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+ TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+ TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+ set +e
+ verbose_docker rm --force "local-airflow-manifest" >/dev/null 2>&1
+ if ! verbose_docker create --name "local-airflow-manifest" \
+ "${AIRFLOW_LOCAL_MANIFEST_IMAGE}" >/dev/null 2>&1 ; then
+ echo
+ echo "Local docker not available"
Review comment:
Or similar - "image" being the key bit to add here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7656:
[AIRFLOW-7013] Automated check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
> Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
> The diff coverage is `n/a`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7656 +/- ##
==========================================
- Coverage 86.98% 86.49% -0.49%
==========================================
Files 904 904
Lines 43736 43736
==========================================
- Hits 38045 37831 -214
- Misses 5691 5905 +214
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
| [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
| [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
| [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
| [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated
check if Breeze image needs to be pulled
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597332080
All should be good now!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services