You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/04/05 00:28:02 UTC
[GitHub] [airflow] brandonwillard opened a new pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
brandonwillard opened a new pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150
This PR introduces a new config option, `kubernetes.extra_volume_mounts`, that allows users to specify multiple Kubernetes volumes to be mounted in each generated worker pod.
This PR replaces #7423 (moved to my personal fork).
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r407746710
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
airflow_local_settings.py file is intended only for logs currently. I am not sure if we want to mix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
dimberman commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r405065696
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
Does a user have to do all of these escapes or can they just supply a dict? I'd rather not have people supplying strings this gnarly if possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
dimberman commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r407730090
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
@kaxil what do you think? should we add this to the airflow_local_settings.py so users can define python dicts?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
dimberman commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r405066226
##########
File path: airflow/executors/kubernetes_executor.py
##########
@@ -241,6 +245,69 @@ def _get_security_context_val(self, scontext: str) -> Union[str, int]:
else:
return int(val)
+ def _parse_extra_volume_mounts(self) -> Dict[str, Dict[str, Any]]:
+ res: Dict[str, Dict[str, Any]] = {}
+
+ extra_volume_mounts = conf.get(self.kubernetes_section, 'extra_volume_mounts')
+
+ if extra_volume_mounts: # pylint: disable=too-many-nested-blocks
+
+ try:
+ res = json.loads(extra_volume_mounts)
+ except JSONDecodeError as e:
+ raise AirflowConfigException(
+ 'Error parsing config option'
+ ' `extra_volume_mounts`: {}.'.format(e))
+
+ for pvc_name, pvc_settings in res.items():
+ if 'mount_path' not in pvc_settings:
Review comment:
Can you please create another helper function to handle the iterations of this for loop? There's a lot of nesting happening here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/68d1714f296989b7aad1a04b75dc033e76afb747&el=desc) will **increase** coverage by `21.42%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
===========================================
+ Coverage 66.39% 87.81% +21.42%
===========================================
Files 935 945 +10
Lines 45170 50509 +5339
===========================================
+ Hits 29990 44357 +14367
+ Misses 15180 6152 -9028
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+46.83%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+81.29%)` | :arrow_up: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-2.86%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.29%)` | :arrow_down: |
| [airflow/hooks/base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9iYXNlX2hvb2sucHk=) | `86.66% <0.00%> (-0.84%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/logs.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9sb2dzLnB5) | `95.23% <0.00%> (-0.60%)` | :arrow_down: |
| [airflow/providers/grpc/hooks/grpc.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ3JwYy9ob29rcy9ncnBjLnB5) | `91.54% <0.00%> (-0.52%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/batch\_client.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9iYXRjaF9jbGllbnQucHk=) | `96.09% <0.00%> (-0.52%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-0.33%)` | :arrow_down: |
| [airflow/utils/trigger\_rule.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy90cmlnZ2VyX3J1bGUucHk=) | `100.00% <0.00%> (ø)` | |
| ... and [560 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [68d1714...8d1cbf4](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8150: [AIRFLOW-3126] Add
option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
dimberman commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r407776611
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
@mik-laj we use the airflow_local_settings.py for the pod_mutation_hook https://github.com/apache/airflow/blob/8465d66f05baeb73dd4479b019515c069444616e/airflow/settings.py#L107 That's why I was thinking it might make sense.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
dimberman commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r407776611
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
@mik-laj we use the airflow_local_settings.py for the pod_mutation_hook https://github.com/apache/airflow/blob/8465d66f05baeb73dd4479b019515c069444616e/airflow/settings.py#L107. That's why I was thinking it might make sense.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] brandonwillard commented on a change in pull request
#8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
brandonwillard commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r407683706
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
Do you want me to add something specific to the documentation before resolving this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8150:
[AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#issuecomment-609113012
# [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=h1) Report
> Merging [#8150](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1982c3fdca1f04cfc41fc5b5e285d8f01c6b76ab&el=desc) will **increase** coverage by `0.84%`.
> The diff coverage is `100.00%`.
[](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #8150 +/- ##
==========================================
+ Coverage 86.97% 87.81% +0.84%
==========================================
Files 927 945 +18
Lines 44963 50509 +5546
==========================================
+ Hits 39108 44357 +5249
- Misses 5855 6152 +297
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMva3ViZXJuZXRlc19leGVjdXRvci5weQ==) | `63.91% <100.00%> (+7.04%)` | :arrow_up: |
| [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.35% <100.00%> (+0.04%)` | :arrow_up: |
| [...contrib/operators/gcp\_translate\_speech\_operator.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNsYXRlX3NwZWVjaF9vcGVyYXRvci5weQ==) | `75.00% <0.00%> (-25.00%)` | :arrow_down: |
| [airflow/contrib/hooks/gcp\_api\_base\_hook.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF9hcGlfYmFzZV9ob29rLnB5) | `88.88% <0.00%> (-11.12%)` | :arrow_down: |
| [airflow/providers/google/ads/hooks/ads.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Fkcy9ob29rcy9hZHMucHk=) | `74.52% <0.00%> (-4.05%)` | :arrow_down: |
| [airflow/providers/amazon/aws/hooks/athena.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9hdGhlbmEucHk=) | `64.07% <0.00%> (-3.11%)` | :arrow_down: |
| [airflow/secrets/base\_secrets.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWNyZXRzL2Jhc2Vfc2VjcmV0cy5weQ==) | `91.30% <0.00%> (-2.82%)` | :arrow_down: |
| [airflow/models/renderedtifields.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvcmVuZGVyZWR0aWZpZWxkcy5weQ==) | `93.22% <0.00%> (-2.62%)` | :arrow_down: |
| [airflow/providers/amazon/aws/sensors/emr\_base.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9iYXNlLnB5) | `90.00% <0.00%> (-1.67%)` | :arrow_down: |
| [...rflow/providers/amazon/aws/sensors/emr\_job\_flow.py](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9zZW5zb3JzL2Vtcl9qb2JfZmxvdy5weQ==) | `94.87% <0.00%> (-1.43%)` | :arrow_down: |
| ... and [116 more](https://codecov.io/gh/apache/airflow/pull/8150/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=footer). Last update [1982c3f...2aa2c86](https://codecov.io/gh/apache/airflow/pull/8150?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] brandonwillard commented on a change in pull request
#8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
brandonwillard commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r405158754
##########
File path: airflow/config_templates/default_airflow.cfg
##########
@@ -881,6 +881,15 @@ logs_volume_subpath =
# A shared volume claim for the logs
logs_volume_claim =
+# Extra volumes to be mounted in worker pods. Volumes are specified as
+# keys in a JSON object with nested JSON values specifying each volume's
+# options. Recognized options are `claim_name` or `secret_name`
+# (required), `mount_path` (required), `read_only` (boolean, default
+# null), `sub_path` (default null), `secret_key` (string, default null),
+# `secret_mode` (string, default null).
+# Example: extra_volume_mounts = "{{{{\"secret_vol\": {{{{\"secret_name\": \"some-secret\", \"mount_path\": \"/dir1\", \"sub_path\": \"subpath1\", \"secret_mode\": \"440\"}}}}, \"pvc\": {{{{\"claim_name\": \"some-pvc\", \"mount_path\": \"/dir2\"}}}}}}}}"
Review comment:
To the user, the value in the `.cfg` file needs to be valid JSON/a `dict`, but, yeah, it's very gnarly here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] brandonwillard commented on a change in pull request
#8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
Posted by GitBox <gi...@apache.org>.
brandonwillard commented on a change in pull request #8150: [AIRFLOW-3126] Add option to specify additional K8s volumes
URL: https://github.com/apache/airflow/pull/8150#discussion_r405158858
##########
File path: airflow/executors/kubernetes_executor.py
##########
@@ -241,6 +245,69 @@ def _get_security_context_val(self, scontext: str) -> Union[str, int]:
else:
return int(val)
+ def _parse_extra_volume_mounts(self) -> Dict[str, Dict[str, Any]]:
+ res: Dict[str, Dict[str, Any]] = {}
+
+ extra_volume_mounts = conf.get(self.kubernetes_section, 'extra_volume_mounts')
+
+ if extra_volume_mounts: # pylint: disable=too-many-nested-blocks
+
+ try:
+ res = json.loads(extra_volume_mounts)
+ except JSONDecodeError as e:
+ raise AirflowConfigException(
+ 'Error parsing config option'
+ ' `extra_volume_mounts`: {}.'.format(e))
+
+ for pvc_name, pvc_settings in res.items():
+ if 'mount_path' not in pvc_settings:
Review comment:
Yeah, no problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services