You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/03/18 00:57:02 UTC
[GitHub] [airflow] kaxil opened a new pull request #7753: [AIRFLOW-5705]
Make AwsSsmSecretsBackend consistent with VaultBackend
kaxil opened a new pull request #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753
Make this PR consistent with https://github.com/apache/airflow/pull/7741 to remove the redundant **AIRFLOW_CONN** from secret key
and add more tests
---
Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Commit message/PR title starts with `[AIRFLOW-NNNN]`. AIRFLOW-NNNN = JIRA ID<sup>*</sup>
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
<sup>*</sup> For document-only changes commit message can start with `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395314615
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -37,34 +38,37 @@ class AwsSsmSecretsBackend(BaseSecretsBackend, LoggingMixin):
[secrets]
backend = airflow.providers.amazon.aws.secrets.ssm.AwsSsmSecretsBackend
- backend_kwargs = {"prefix": "/airflow", "profile_name": null}
-
- For example, if ssm path is ``/airflow/AIRFLOW_CONN_SMTP_DEFAULT``, this would be accessible if you
- provide ``{"prefix": "/airflow"}`` and request conn_id ``smtp_default``.
+ backend_kwargs = {"connections_prefix": "/airflow/connections", "profile_name": null}
+ For example, if ssm path is ``/airflow/connections/smtp_default``, this would be accessible
+ if you provide ``{"connections_prefix": "/airflow"}`` and request conn_id ``smtp_default``.
Review comment:
```suggestion
if you provide ``{"connections_prefix": "/airflow/connections"}`` and request conn_id ``smtp_default``.
```
I think
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#issuecomment-600392755
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=h1) Report
> Merging [#7753](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/029c84e5527b6db6bdbdbe026f455da325bedef3&el=desc) will **decrease** coverage by `0.84%`.
> The diff coverage is `67.77%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7753/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7753 +/- ##
==========================================
- Coverage 86.96% 86.11% -0.85%
==========================================
Files 920 921 +1
Lines 44436 44462 +26
==========================================
- Hits 38644 38290 -354
- Misses 5792 6172 +380
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/providers/hashicorp/secrets/vault.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaGFzaGljb3JwL3NlY3JldHMvdmF1bHQucHk=) | `62.33% <62.33%> (ø)` | |
| [airflow/providers/amazon/aws/secrets/ssm.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZWNyZXRzL3NzbS5weQ==) | `100.00% <100.00%> (+5.71%)` | :arrow_up: |
| [airflow/secrets/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL19faW5pdF9fLnB5) | `97.61% <100.00%> (-0.06%)` | :arrow_down: |
| [airflow/secrets/environment\_variables.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Vudmlyb25tZW50X3ZhcmlhYmxlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=footer). Last update [029c84e...9655c92](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dstandish commented on a change in pull request #7753:
[AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
dstandish commented on a change in pull request #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r394057083
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -21,9 +21,10 @@
from typing import List, Optional
import boto3
+from cached_property import cached_property
from airflow.models import Connection
-from airflow.secrets import CONN_ENV_PREFIX, BaseSecretsBackend
Review comment:
`CONN_ENV_PREFIX` should probably be moved to `airflow/secrets/environment_variables.py` since it is no longer a shared pattern, but something specific to the env vars backend...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395316283
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -37,34 +38,37 @@ class AwsSsmSecretsBackend(BaseSecretsBackend, LoggingMixin):
[secrets]
backend = airflow.providers.amazon.aws.secrets.ssm.AwsSsmSecretsBackend
- backend_kwargs = {"prefix": "/airflow", "profile_name": null}
-
- For example, if ssm path is ``/airflow/AIRFLOW_CONN_SMTP_DEFAULT``, this would be accessible if you
- provide ``{"prefix": "/airflow"}`` and request conn_id ``smtp_default``.
+ backend_kwargs = {"connections_prefix": "/airflow/connections", "profile_name": null}
+ For example, if ssm path is ``/airflow/connections/smtp_default``, this would be accessible
+ if you provide ``{"connections_prefix": "/airflow"}`` and request conn_id ``smtp_default``.
Review comment:
fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395001497
##########
File path: tests/providers/apache/hive/hooks/test_hive.py
##########
@@ -33,7 +33,7 @@
from airflow.models.dag import DAG
from airflow.providers.apache.hive.hooks.hive import HiveCliHook, HiveMetastoreHook, HiveServer2Hook
from airflow.providers.apache.hive.operators.hive import HiveOperator
-from airflow.secrets import CONN_ENV_PREFIX
+from airflow.secrets.environment_variables import CONN_ENV_PREFIX
Review comment:
Based on https://github.com/apache/airflow/pull/7753#discussion_r394057083
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#issuecomment-600392755
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=h1) Report
> Merging [#7753](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/029c84e5527b6db6bdbdbe026f455da325bedef3&el=desc) will **decrease** coverage by `0.84%`.
> The diff coverage is `67.77%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7753/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7753 +/- ##
==========================================
- Coverage 86.96% 86.11% -0.85%
==========================================
Files 920 921 +1
Lines 44436 44462 +26
==========================================
- Hits 38644 38290 -354
- Misses 5792 6172 +380
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/providers/hashicorp/secrets/vault.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvaGFzaGljb3JwL3NlY3JldHMvdmF1bHQucHk=) | `62.33% <62.33%> (ø)` | |
| [airflow/providers/amazon/aws/secrets/ssm.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZWNyZXRzL3NzbS5weQ==) | `100.00% <100.00%> (+5.71%)` | :arrow_up: |
| [airflow/secrets/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL19faW5pdF9fLnB5) | `97.61% <100.00%> (-0.06%)` | :arrow_down: |
| [airflow/secrets/environment\_variables.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Vudmlyb25tZW50X3ZhcmlhYmxlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=footer). Last update [029c84e...9655c92](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395316253
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -37,34 +38,37 @@ class AwsSsmSecretsBackend(BaseSecretsBackend, LoggingMixin):
[secrets]
backend = airflow.providers.amazon.aws.secrets.ssm.AwsSsmSecretsBackend
- backend_kwargs = {"prefix": "/airflow", "profile_name": null}
-
- For example, if ssm path is ``/airflow/AIRFLOW_CONN_SMTP_DEFAULT``, this would be accessible if you
- provide ``{"prefix": "/airflow"}`` and request conn_id ``smtp_default``.
+ backend_kwargs = {"connections_prefix": "/airflow/connections", "profile_name": null}
+ For example, if ssm path is ``/airflow/connections/smtp_default``, this would be accessible
+ if you provide ``{"connections_prefix": "/airflow"}`` and request conn_id ``smtp_default``.
"""
- def __init__(self, prefix: str = '/airflow', profile_name: Optional[str] = None, **kwargs):
- self._prefix = prefix
+ def __init__(
+ self, connections_prefix: str = '/airflow/connections',
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7753:
[AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r394059863
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -21,9 +21,10 @@
from typing import List, Optional
import boto3
+from cached_property import cached_property
from airflow.models import Connection
-from airflow.secrets import CONN_ENV_PREFIX, BaseSecretsBackend
Review comment:
Fixed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7753: [AIRFLOW-5705] Make
AwsSsmSecretsBackend consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#issuecomment-600392755
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=h1) Report
> Merging [#7753](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/49998edd2ff0b64fd1771138fc7d8e835c564a47?src=pr&el=desc) will **decrease** coverage by `22.28%`.
> The diff coverage is `100%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7753/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7753 +/- ##
==========================================
- Coverage 86.99% 64.7% -22.29%
==========================================
Files 915 914 -1
Lines 44198 44186 -12
==========================================
- Hits 38451 28592 -9859
- Misses 5747 15594 +9847
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/providers/amazon/aws/secrets/ssm.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZWNyZXRzL3NzbS5weQ==) | `100% <100%> (+5.71%)` | :arrow_up: |
| [airflow/secrets/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL19faW5pdF9fLnB5) | `92.85% <100%> (-4.82%)` | :arrow_down: |
| [airflow/secrets/environment\_variables.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Vudmlyb25tZW50X3ZhcmlhYmxlcy5weQ==) | `100% <100%> (ø)` | :arrow_up: |
| [...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/hooks/vertica\_hook.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3ZlcnRpY2FfaG9vay5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/sensors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvX19pbml0X18ucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/hooks/mssql\_hook.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9tc3NxbF9ob29rLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/hooks/webhdfs\_hook.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy93ZWJoZGZzX2hvb2sucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/contrib/sensors/emr\_base\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvZW1yX2Jhc2Vfc2Vuc29yLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| ... and [493 more](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=footer). Last update [49998ed...6da2cde](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395314905
##########
File path: airflow/providers/amazon/aws/secrets/ssm.py
##########
@@ -37,34 +38,37 @@ class AwsSsmSecretsBackend(BaseSecretsBackend, LoggingMixin):
[secrets]
backend = airflow.providers.amazon.aws.secrets.ssm.AwsSsmSecretsBackend
- backend_kwargs = {"prefix": "/airflow", "profile_name": null}
-
- For example, if ssm path is ``/airflow/AIRFLOW_CONN_SMTP_DEFAULT``, this would be accessible if you
- provide ``{"prefix": "/airflow"}`` and request conn_id ``smtp_default``.
+ backend_kwargs = {"connections_prefix": "/airflow/connections", "profile_name": null}
+ For example, if ssm path is ``/airflow/connections/smtp_default``, this would be accessible
+ if you provide ``{"connections_prefix": "/airflow"}`` and request conn_id ``smtp_default``.
"""
- def __init__(self, prefix: str = '/airflow', profile_name: Optional[str] = None, **kwargs):
- self._prefix = prefix
+ def __init__(
+ self, connections_prefix: str = '/airflow/connections',
Review comment:
```suggestion
self,
connections_prefix: str = '/airflow/connections',
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil merged pull request #7753: [AIRFLOW-5705] Make
AwsSsmSecretsBackend consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7753:
[AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend
consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #7753: [AIRFLOW-5705][depends on AIRFLOW-7076] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#discussion_r395000868
##########
File path: tests/providers/apache/hive/hooks/test_hive.py
##########
@@ -33,7 +33,7 @@
from airflow.models.dag import DAG
from airflow.providers.apache.hive.hooks.hive import HiveCliHook, HiveMetastoreHook, HiveServer2Hook
from airflow.providers.apache.hive.operators.hive import HiveOperator
-from airflow.secrets import CONN_ENV_PREFIX
+from airflow.secrets.environment_variables import CONN_ENV_PREFIX
Review comment:
?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7753:
[AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7753: [AIRFLOW-5705] Make AwsSsmSecretsBackend consistent with VaultBackend
URL: https://github.com/apache/airflow/pull/7753#issuecomment-600392755
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=h1) Report
> Merging [#7753](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/49998edd2ff0b64fd1771138fc7d8e835c564a47&el=desc) will **decrease** coverage by `0.77%`.
> The diff coverage is `100.00%`.
[![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7753/graphs/tree.svg?width=650&height=150&src=pr&token=WdLKlKHOAU)](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7753 +/- ##
==========================================
- Coverage 86.99% 86.21% -0.78%
==========================================
Files 915 915
Lines 44198 44199 +1
==========================================
- Hits 38451 38108 -343
- Misses 5747 6091 +344
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/providers/amazon/aws/secrets/ssm.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZWNyZXRzL3NzbS5weQ==) | `100.00% <100.00%> (+5.71%)` | :arrow_up: |
| [airflow/secrets/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL19faW5pdF9fLnB5) | `97.61% <100.00%> (-0.06%)` | :arrow_down: |
| [airflow/secrets/environment\_variables.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Vudmlyb25tZW50X3ZhcmlhYmxlcy5weQ==) | `100.00% <100.00%> (ø)` | |
| [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.51% <0.00%> (-72.16%)` | :arrow_down: |
| [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: |
| [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: |
| [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: |
| ... and [15 more](https://codecov.io/gh/apache/airflow/pull/7753/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=footer). Last update [49998ed...6da2cde](https://codecov.io/gh/apache/airflow/pull/7753?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services