You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/03/08 14:44:48 UTC

[GitHub] [airflow] potiuk opened a new pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

potiuk opened a new pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656
 
 
   ---
   Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
   
   Make sure to mark the boxes below before creating PR: [x]
   
   - [x] Description above provides context of the change
   - [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
   - [x] Unit tests coverage for changes (not needed for documentation changes)
   - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
   - [x] Relevant documentation is updated including usage instructions.
   - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
   
   <sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
   
   ---
   In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
   Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390568970
 
 

 ##########
 File path: scripts/ci/_utils.sh
 ##########
 @@ -552,6 +561,85 @@ function set_current_image_variables {
     fi
 }
 
+function build_image_manifest() {
+    verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+    verbose_docker build \
+    --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+    --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+    -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+    TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+    TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+    set +e
+    verbose_docker rm --force "local-airflow-manifest"  >/dev/null 2>&1
+    if ! verbose_docker create --name "local-airflow-manifest" \
+        "${AIRFLOW_LOCAL_MANIFEST_IMAGE}"  >/dev/null 2>&1 ; then
+        echo
+        echo "Local docker not available"
 
 Review comment:
   Added full description + a bit nicer output showing pull progress as well as step of the docker build during pre-commit.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
   > Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.64%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7656      +/-   ##
   ==========================================
   - Coverage   86.98%   86.34%   -0.65%     
   ==========================================
     Files         904      904              
     Lines       43736    43736              
   ==========================================
   - Hits        38045    37764     -281     
   - Misses       5691     5972     +281     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
   | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
   | [airflow/api/auth/backend/kerberos\_auth.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9hcGkvYXV0aC9iYWNrZW5kL2tlcmJlcm9zX2F1dGgucHk=) | `28.16% <0.00%> (-54.93%)` | :arrow_down: |
   | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
   | [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0.00%> (-45.66%)` | :arrow_down: |
   | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
   | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
   | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
   | ... and [10 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390333381
 
 

 ##########
 File path: scripts/ci/_utils.sh
 ##########
 @@ -552,6 +561,85 @@ function set_current_image_variables {
     fi
 }
 
+function build_image_manifest() {
+    verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+    verbose_docker build \
+    --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+    --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+    -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+    TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+    TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+    set +e
+    verbose_docker rm --force "local-airflow-manifest"  >/dev/null 2>&1
+    if ! verbose_docker create --name "local-airflow-manifest" \
+        "${AIRFLOW_LOCAL_MANIFEST_IMAGE}"  >/dev/null 2>&1 ; then
+        echo
+        echo "Local docker not available"
 
 Review comment:
   ```suggestion
           echo "Local airflow CI docker image not available"
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390460517
 
 

 ##########
 File path: scripts/ci/_utils.sh
 ##########
 @@ -552,6 +561,85 @@ function set_current_image_variables {
     fi
 }
 
+function build_image_manifest() {
+    verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+    verbose_docker build \
+    --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+    --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+    -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+    TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+    TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+    set +e
+    verbose_docker rm --force "local-airflow-manifest"  >/dev/null 2>&1
+    if ! verbose_docker create --name "local-airflow-manifest" \
+        "${AIRFLOW_LOCAL_MANIFEST_IMAGE}"  >/dev/null 2>&1 ; then
+        echo
+        echo "Local docker not available"
 
 Review comment:
   > Can you add some docs/comments in the PR somewhere about
   > 
   > 1. what this is doing -- I'm having trouble following the logic; and
   > 2. why (your comment in the PR should go in the script somewhere)
   
   Good point. I added comments describing both.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
   > Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7656      +/-   ##
   ==========================================
   - Coverage   86.98%   86.49%   -0.49%     
   ==========================================
     Files         904      904              
     Lines       43736    43736              
   ==========================================
   - Hits        38045    37831     -214     
   - Misses       5691     5905     +214     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
   | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
   | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
   | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
   | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
   | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
   | [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
   | [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
   | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk merged pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597575192
 
 
   Hey @ashb -> are you OK with those comments ?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
   > Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7656      +/-   ##
   ==========================================
   - Coverage   86.98%   86.49%   -0.49%     
   ==========================================
     Files         904      904              
     Lines       43736    43736              
   ==========================================
   - Hits        38045    37831     -214     
   - Misses       5691     5905     +214     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
   | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
   | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
   | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
   | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
   | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
   | [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
   | [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
   | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-596213504
 
 
   This one implements automated check if it is worth to pull image rather than build it locally. When the image is changed a lot (for example with buster) it makes very little sense to rebuild the image, it's better to pull it. 
   
   This change will automatically detect if it is worth to build or pull&build image when you reach the "need rebuild phase". 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#discussion_r390333670
 
 

 ##########
 File path: scripts/ci/_utils.sh
 ##########
 @@ -552,6 +561,85 @@ function set_current_image_variables {
     fi
 }
 
+function build_image_manifest() {
+    verbose_docker inspect "${AIRFLOW_IMAGE}" > "manifests/${AIRFLOW_BASE_TAG}.json"
+    verbose_docker build \
+    --build-arg AIRFLOW_BASE_TAG="${AIRFLOW_BASE_TAG}" \
+    --tag="${AIRFLOW_LOCAL_MANIFEST_IMAGE}" \
+    -f- . <<EOF
+ARG AIRFLOW_BASE_TAG
+FROM scratch
+
+COPY "manifests/${AIRFLOW_BASE_TAG}.json" .
+
+CMD ""
+EOF
+}
+
+function get_local_image_info() {
+    TMP_MANIFEST_LOCAL_JSON=$(mktemp)
+    TMP_MANIFEST_LOCAL_SHA=$(mktemp)
+    set +e
+    verbose_docker rm --force "local-airflow-manifest"  >/dev/null 2>&1
+    if ! verbose_docker create --name "local-airflow-manifest" \
+        "${AIRFLOW_LOCAL_MANIFEST_IMAGE}"  >/dev/null 2>&1 ; then
+        echo
+        echo "Local docker not available"
 
 Review comment:
   Or similar - "image" being the key bit to add here.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597323139
 
 
   # [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=h1) Report
   > Merging [#7656](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/fcfc2aa7317916b402340e223b4431f87d9d6d6f&el=desc) will **decrease** coverage by `0.48%`.
   > The diff coverage is `n/a`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7656/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##           master    #7656      +/-   ##
   ==========================================
   - Coverage   86.98%   86.49%   -0.49%     
   ==========================================
     Files         904      904              
     Lines       43736    43736              
   ==========================================
   - Hits        38045    37831     -214     
   - Misses       5691     5905     +214     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
   | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
   | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
   | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
   | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: |
   | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: |
   | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: |
   | [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `52.94% <0.00%> (-32.36%)` | :arrow_down: |
   | [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-15.50%)` | :arrow_down: |
   | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/7656/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=footer). Last update [fcfc2aa...115badc](https://codecov.io/gh/apache/airflow/pull/7656?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [airflow] potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled

Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7656: [AIRFLOW-7013] Automated check if Breeze image needs to be pulled
URL: https://github.com/apache/airflow/pull/7656#issuecomment-597332080
 
 
   All should be good now!

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services