You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/06 19:42:08 UTC
[GitHub] [airflow] lewismc opened a new pull request #13519: Upodate chart README with section on Airflow configuration
lewismc opened a new pull request #13519:
URL: https://github.com/apache/airflow/pull/13519
Being new to the Helm Chart, I asked a [question](https://lists.apache.org/thread.html/r157160eab6e8ae714ab0dfb0a22125059fe1bffada871fb5e24f3bfa%40%3Cusers.airflow.apache.org%3E) on user@airflow.a.o. I finally discovered that the Airflow configuration parameters can be entered in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml).
This simple PR should hopefully help anyone else in a similar situation to me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13519: Update chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13519:
URL: https://github.com/apache/airflow/pull/13519#discussion_r553074438
##########
File path: chart/README.md
##########
@@ -32,6 +32,47 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml) under the `config` key (around line 650 of `values.yaml`). The following code demonstrates how one would deny webserver users from viewing the config from within the webserver application. See the bottom line of the example...
Review comment:
```suggestion
All Airflow configuration parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml) under the `config` key . The following code demonstrates how one would deny webserver users from viewing the config from within the webserver application. See the bottom line of the example:
```
##########
File path: chart/README.md
##########
@@ -32,6 +32,47 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml) under the `config` key (around line 650 of `values.yaml`). The following code demonstrates how one would deny webserver users from viewing the config from within the webserver application. See the bottom line of the example...
+
+```
Review comment:
```suggestion
```yaml
```
##########
File path: chart/README.md
##########
@@ -32,6 +32,47 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml) under the `config` key (around line 650 of `values.yaml`). The following code demonstrates how one would deny webserver users from viewing the config from within the webserver application. See the bottom line of the example...
+
+```
+# Config settings to go into the mounted airflow.cfg
+#
+# Please note that these values are passed through the `tpl` function, so are
+# all subject to being rendered as go templates. If you need to include a
+# literal `{{` in a value, it must be expressed like this:
+#
+# a: '{{ "{{ not a template }}" }}'
+#
+# yamllint disable rule:line-length
+config:
+ core:
+ dags_folder: '{{ include "airflow_dags" . }}'
+ load_examples: 'False'
+ executor: '{{ .Values.executor }}'
+ # For Airflow 1.10, backward compatibility
+ colored_console_log: 'False'
+ remote_logging: '{{- ternary "True" "False" .Values.elasticsearch.enabled }}'
+ # Authentication backend used for the experimental API
+ api:
+ auth_backend: airflow.api.auth.backend.deny_all
+ logging:
+ remote_logging: '{{- ternary "True" "False" .Values.elasticsearch.enabled }}'
+ colored_console_log: 'False'
+ logging_level: DEBUG
+ metrics:
+ statsd_on: '{{ ternary "True" "False" .Values.statsd.enabled }}'
+ statsd_port: 9125
+ statsd_prefix: airflow
+ statsd_host: '{{ printf "%s-statsd" .Release.Name }}'
+ webserver:
+ enable_proxy_fix: 'True'
+ expose_config: 'False' <<<<<<<<<< BY DEFAULT THIS IS 'True' BUT WE CHANGE IT TO 'False' PRIOR TO INSTALLING THE CHART
Review comment:
```suggestion
expose_config: 'False' # <<<<<<<<<< BY DEFAULT THIS IS 'True' BUT WE CHANGE IT TO 'False' PRIOR TO INSTALLING THE CHART
```
##########
File path: chart/README.md
##########
@@ -32,6 +32,47 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml) under the `config` key (around line 650 of `values.yaml`). The following code demonstrates how one would deny webserver users from viewing the config from within the webserver application. See the bottom line of the example...
+
+```
+# Config settings to go into the mounted airflow.cfg
+#
+# Please note that these values are passed through the `tpl` function, so are
+# all subject to being rendered as go templates. If you need to include a
+# literal `{{` in a value, it must be expressed like this:
+#
+# a: '{{ "{{ not a template }}" }}'
+#
+# yamllint disable rule:line-length
+config:
+ core:
+ dags_folder: '{{ include "airflow_dags" . }}'
+ load_examples: 'False'
+ executor: '{{ .Values.executor }}'
+ # For Airflow 1.10, backward compatibility
+ colored_console_log: 'False'
+ remote_logging: '{{- ternary "True" "False" .Values.elasticsearch.enabled }}'
+ # Authentication backend used for the experimental API
+ api:
+ auth_backend: airflow.api.auth.backend.deny_all
+ logging:
+ remote_logging: '{{- ternary "True" "False" .Values.elasticsearch.enabled }}'
+ colored_console_log: 'False'
+ logging_level: DEBUG
+ metrics:
+ statsd_on: '{{ ternary "True" "False" .Values.statsd.enabled }}'
+ statsd_port: 9125
+ statsd_prefix: airflow
+ statsd_host: '{{ printf "%s-statsd" .Release.Name }}'
+ webserver:
+ enable_proxy_fix: 'True'
+ expose_config: 'False' <<<<<<<<<< BY DEFAULT THIS IS 'True' BUT WE CHANGE IT TO 'False' PRIOR TO INSTALLING THE CHART
+```
+
+Generally speaking, it is useful to familiarize ones self with the Airflow configuration prior to installing and deploying the service.
Review comment:
```suggestion
Generally speaking, it is useful to familiarize oneself with the Airflow configuration prior to installing and deploying the service.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #13519: Upodate chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #13519:
URL: https://github.com/apache/airflow/pull/13519#issuecomment-755580632
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, itโs a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better ๐.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13519: Update chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13519:
URL: https://github.com/apache/airflow/pull/13519#discussion_r553031467
##########
File path: chart/README.md
##########
@@ -32,6 +32,10 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml). You may wish to consult these prior to Chart installation.
+
Review comment:
The description is not clear enough, it also does not tell which key needs to be set, all configs need to go under `config` key:
https://github.com/apache/airflow/blob/9c75ea3c14b71d2f96d997aeef68c764c7d2984c/chart/values.yaml#L643
Would be better to give an example too
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] lewismc commented on a change in pull request #13519: Update chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
lewismc commented on a change in pull request #13519:
URL: https://github.com/apache/airflow/pull/13519#discussion_r553068789
##########
File path: chart/README.md
##########
@@ -32,6 +32,10 @@ cluster using the [Helm](https://helm.sh) package manager.
- Helm 2.11+ or Helm 3.0+
- PV provisioner support in the underlying infrastructure
+## Configuring Airflow
+
+All Airflow configuation parameters (equivalent of `airflow.cfg`) are stored in [values.yaml](https://github.com/apache/airflow/blob/master/chart/values.yaml). You may wish to consult these prior to Chart installation.
+
Review comment:
Thank you for the suggestion @kaxil I appreciate that. I've augmented the PR to accommodate.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #13519: Update chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #13519:
URL: https://github.com/apache/airflow/pull/13519#issuecomment-756106875
Awesome work, congrats on your first merged pull request!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #13519: Update chart README with section on Airflow configuration
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #13519:
URL: https://github.com/apache/airflow/pull/13519
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org