You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/01/03 05:57:02 UTC
[GitHub] [airflow] kaxil opened a new pull request #7015: [AIRFLOW-6436]
Create & Automate docs on Airflow Configs
kaxil opened a new pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015
This PR aims to automate the creation of Config docs and add pre-commit hooks that would also serve as tests.
**Preview**:
- **Layout 1**: https://near-cherry.surge.sh/configurations-ref.html
- **Layout 2**: https://flowery-ray.surge.sh/configurations-ref.html (Similar to our CLI doc)
Also, add a structure to the documentation by adding the following sections:
- Description
- Example
- Type
- Default
- for each config
---
Link to JIRA issue: https://issues.apache.org/jira/browse/AIRFLOW-6436
- [x] Description above provides context of the change
- [x] Commit message starts with `[AIRFLOW-NNNN]`, where AIRFLOW-NNNN = JIRA ID*
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
(*) For document-only changes, no JIRA issue is needed. Commit message starts `[AIRFLOW-XXXX]`.
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570472178
This PR needs a bit of polishing but its overall logic or structure would remain same.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r362899422
##########
File path: .github/mergeable.yml
##########
@@ -50,3 +50,8 @@ mergeable:
changed:
file: 'package.json'
files: ['yarn.lock']
+ # If config.yml is updated, so should default_airflow.cfg
+ - do: dependent
Review comment:
I was too late (just started to write this comment) I think this might be not good check. If someone modifies default_airflow.cfg and changes value or add space, probot will complain. I guess pre-commit-config should be more than enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570645822
Going to follow this up with some polishing up PRs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] DENjjA commented on issue #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
DENjjA commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-585617332
Hi!
Latest commit - https://github.com/apache/airflow/commit/9b5dbf9886df3977b24c95cadb99bbece9f8ce4b#diff-bbf16e7665ac448883f2ceeb40db35cdR444
in airflow 1.10.8 (1.10.9)
does not allow changing worker concurrency through CLI interface
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7015: [AIRFLOW-6436]
Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.94%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.08% -0.95%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32447 -328
- Misses 5770 6143 +373
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.65% <0%> (-38.78%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.42%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.42% -0.43%
==========================================
Files 679 680 +1
Lines 38536 38725 +189
==========================================
- Hits 32698 32692 -6
- Misses 5838 6033 +195
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| [airflow/utils/helpers.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `74.5% <0%> (-7.19%)` | :arrow_down: |
| [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `80.95% <0%> (-7.05%)` | :arrow_down: |
| ... and [60 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r362900077
##########
File path: .github/mergeable.yml
##########
@@ -50,3 +50,8 @@ mergeable:
changed:
file: 'package.json'
files: ['yarn.lock']
+ # If config.yml is updated, so should default_airflow.cfg
+ - do: dependent
Review comment:
But at least now that we have got this merged, need configs can directly go to yml with description :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.94%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.08% -0.95%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32447 -328
- Misses 5770 6143 +373
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.65% <0%> (-38.78%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil edited a comment on issue #7015: [AIRFLOW-6436]
Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570645822
Going to follow this up with some polishing up PRs.
So, please feel free to add comments on this PR, I will take care of all the comments and fix them in the follow-up PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r362901991
##########
File path: .github/mergeable.yml
##########
@@ -50,3 +50,8 @@ mergeable:
changed:
file: 'package.json'
files: ['yarn.lock']
+ # If config.yml is updated, so should default_airflow.cfg
+ - do: dependent
Review comment:
Withdraw. I see that all of it is generated automatically, so no risk here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r363007496
##########
File path: airflow/utils/config_yaml_to_cfg.py
##########
@@ -0,0 +1,116 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Module to covert Airflow configs in config.yml to default_airflow.cfg file
+"""
+
+import os
+
+import yaml
+
+AIRFLOW_CONFIG_DIR = os.path.join(os.path.dirname(__file__), "..", "config_templates")
+AIRFLOW_DEFAULT_CONFIG = os.path.join(AIRFLOW_CONFIG_DIR, "default_airflow.cfg")
+AIRFLOW_CONFIG_YAML_FILE_PATH = os.path.join(AIRFLOW_CONFIG_DIR, "config.yml")
+FILE_HEADER = """# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+# This is the template for Airflow's default configuration. When Airflow is
+# imported, it looks for a configuration file at $AIRFLOW_HOME/airflow.cfg. If
+# it doesn't exist, Airflow uses this template to generate it by replacing
+# variables in curly braces with their global values from configuration.py.
+
+# Users should not modify this file; they should customize the generated
+# airflow.cfg instead.
+
+
+# ----------------------- TEMPLATE BEGINS HERE -----------------------
+"""
+
+
+def default_config_yaml() -> dict:
Review comment:
Should we really do this when loading this module? Maybe you can create a function that will only be called in specific cases? If it is an executable file then it should have the following code at the end of the file
```python
if __name__ == '__main__':
do_some_magic();
```
Should it also be in the `airflow` directory? In my opinion, it looks better in the `tests` or` scripts` directory.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dimberman commented on issue #7015: [AIRFLOW-6436] Create
& Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570644124
This is fantastic. 10/10 @kaxil!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.78%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.24% -0.79%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32511 -264
- Misses 5770 6079 +309
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.94%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.08% -0.95%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32447 -328
- Misses 5770 6143 +373
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.65% <0%> (-38.78%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570540102
Love it. I will take a look latere. Akls @dimberman -> i think we might want to merge your PR #6983 with this one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r362899805
##########
File path: .github/mergeable.yml
##########
@@ -50,3 +50,8 @@ mergeable:
changed:
file: 'package.json'
files: ['yarn.lock']
+ # If config.yml is updated, so should default_airflow.cfg
+ - do: dependent
Review comment:
Yup, I am going to follow up with a couple more PRs on it for polishing. I will remove it from mergeable in that one
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil merged pull request #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0f777f74b776b45e94615f423f61a8eb0025db83?src=pr&el=desc) will **decrease** coverage by `0.76%`.
> The diff coverage is `60.45%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 84.85% 84.08% -0.77%
==========================================
Files 679 680 +1
Lines 38536 38590 +54
==========================================
- Hits 32698 32447 -251
- Misses 5838 6143 +305
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (ø)` | :arrow_up: |
| [airflow/gcp/example\_dags/example\_gcs.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9nY3AvZXhhbXBsZV9kYWdzL2V4YW1wbGVfZ2NzLnB5) | `100% <ø> (+11.42%)` | :arrow_up: |
| [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: |
| [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.95% <ø> (ø)` | :arrow_up: |
| [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `71.81% <ø> (ø)` | :arrow_up: |
| [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/utils/log/wasb\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvd2FzYl90YXNrX2hhbmRsZXIucHk=) | `42.46% <0%> (ø)` | :arrow_up: |
| [airflow/utils/log/gcs\_task\_handler.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZ2NzX3Rhc2tfaGFuZGxlci5weQ==) | `0% <0%> (ø)` | :arrow_up: |
| [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `85% <100%> (-11.62%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [0f777f7...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.94%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.08% -0.95%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32447 -328
- Misses 5770 6143 +373
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `49.65% <0%> (-38.78%)` | :arrow_down: |
| ... and [13 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ryw commented on issue #7015: [AIRFLOW-6436] Create &
Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
ryw commented on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570657661
Very nice :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#issuecomment-570480731
# [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=h1) Report
> Merging [#7015](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/f391039be9cb5a767f4c66771ba70031210d3e76?src=pr&el=desc) will **decrease** coverage by `0.78%`.
> The diff coverage is `4.44%`.
[](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #7015 +/- ##
==========================================
- Coverage 85.03% 84.24% -0.79%
==========================================
Files 679 680 +1
Lines 38545 38590 +45
==========================================
- Hits 32775 32511 -264
- Misses 5770 6079 +309
```
| [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [airflow/utils/config\_yaml\_to\_cfg.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9jb25maWdfeWFtbF90b19jZmcucHk=) | `0% <0%> (ø)` | |
| [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `91.43% <33.33%> (-1.22%)` | :arrow_down: |
| [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: |
| [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: |
| [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: |
| [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: |
| [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: |
| [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: |
| [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.75% <0%> (-20%)` | :arrow_down: |
| ... and [10 more](https://codecov.io/gh/apache/airflow/pull/7015/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=footer). Last update [f391039...ef62156](https://codecov.io/gh/apache/airflow/pull/7015?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #7015:
[AIRFLOW-6436] Create & Automate docs on Airflow Configs
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #7015: [AIRFLOW-6436] Create & Automate docs on Airflow Configs
URL: https://github.com/apache/airflow/pull/7015#discussion_r363011717
##########
File path: airflow/utils/config_yaml_to_cfg.py
##########
@@ -0,0 +1,116 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Module to covert Airflow configs in config.yml to default_airflow.cfg file
+"""
+
+import os
+
+import yaml
+
+AIRFLOW_CONFIG_DIR = os.path.join(os.path.dirname(__file__), "..", "config_templates")
+AIRFLOW_DEFAULT_CONFIG = os.path.join(AIRFLOW_CONFIG_DIR, "default_airflow.cfg")
+AIRFLOW_CONFIG_YAML_FILE_PATH = os.path.join(AIRFLOW_CONFIG_DIR, "config.yml")
+FILE_HEADER = """# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+
+# This is the template for Airflow's default configuration. When Airflow is
+# imported, it looks for a configuration file at $AIRFLOW_HOME/airflow.cfg. If
+# it doesn't exist, Airflow uses this template to generate it by replacing
+# variables in curly braces with their global values from configuration.py.
+
+# Users should not modify this file; they should customize the generated
+# airflow.cfg instead.
+
+
+# ----------------------- TEMPLATE BEGINS HERE -----------------------
+"""
+
+
+def default_config_yaml() -> dict:
Review comment:
Agree, updating it in https://github.com/apache/airflow/pull/7036
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services