You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by ka...@apache.org on 2020/08/09 14:33:56 UTC
[airflow] branch master updated: Create "Managing variable" in
howto directory (#10241)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/master by this push:
new 183cb8d Create "Managing variable" in howto directory (#10241)
183cb8d is described below
commit 183cb8d56b876d72b21b84db9e218085a7a490d9
Author: Kamil BreguĊa <mi...@users.noreply.github.com>
AuthorDate: Sun Aug 9 16:33:11 2020 +0200
Create "Managing variable" in howto directory (#10241)
---
docs/concepts.rst | 33 +------------------
docs/howto/connection/index.rst | 5 ++-
docs/howto/index.rst | 1 +
docs/howto/variable.rst | 72 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 76 insertions(+), 35 deletions(-)
diff --git a/docs/concepts.rst b/docs/concepts.rst
index a73d7c3..13f887a 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -788,38 +788,7 @@ or if you need to deserialize a json object from the variable :
echo {{ var.json.<variable_name> }}
-Storing Variables in Environment Variables
-------------------------------------------
-
-.. versionadded:: 1.10.10
-
-Airflow Variables can also be created and managed using Environment Variables. The environment variable
-naming convention is :envvar:`AIRFLOW_VAR_{VARIABLE_NAME}`, all uppercase.
-So if your variable key is ``FOO`` then the variable name should be ``AIRFLOW_VAR_FOO``.
-
-For example,
-
-.. code-block:: bash
-
- export AIRFLOW_VAR_FOO=BAR
-
- # To use JSON, store them as JSON strings
- export AIRFLOW_VAR_FOO_BAZ='{"hello":"world"}'
-
-You can use them in your DAGs as:
-
-.. code-block:: python
-
- from airflow.models import Variable
- foo = Variable.get("foo")
- foo_json = Variable.get("foo_baz", deserialize_json=True)
-
-.. note::
-
- Single underscores surround ``VAR``. This is in contrast with the way ``airflow.cfg``
- parameters are stored, where double underscores surround the config section name.
- Variables set using Environment Variables would not appear in the Airflow UI but you will
- be able to use it in your DAG file.
+See :doc:`howto/variable` for details on managing variables.
Branching
=========
diff --git a/docs/howto/connection/index.rst b/docs/howto/connection/index.rst
index 7f4a869..8b56271 100644
--- a/docs/howto/connection/index.rst
+++ b/docs/howto/connection/index.rst
@@ -309,9 +309,8 @@ Securing Connections
--------------------
Airflow uses `Fernet <https://github.com/fernet/spec/>`__ to encrypt passwords in the connection
-configurations stored the metastore database. It guarantees that without the encryption password, Connection Passwords cannot be manipulated or read without the key.
-
-For information on configuring Fernet, look at :ref:`security/fernet`.
+configurations stored the metastore database. It guarantees that without the encryption password, Connection
+Passwords cannot be manipulated or read without the key. For information on configuring Fernet, look at :ref:`security/fernet`.
In addition to retrieving connections from environment variables or the metastore database, you can enable
an secrets backend to retrieve connections. For more details see :doc:`../secrets-backend/index`
diff --git a/docs/howto/index.rst b/docs/howto/index.rst
index a47dd20..1a76b71 100644
--- a/docs/howto/index.rst
+++ b/docs/howto/index.rst
@@ -37,6 +37,7 @@ configuring an Airflow environment.
customize-state-colors-ui
custom-operator
connection/index
+ variable
write-logs
run-behind-proxy
run-with-systemd
diff --git a/docs/howto/variable.rst b/docs/howto/variable.rst
new file mode 100644
index 0000000..ce7bc02
--- /dev/null
+++ b/docs/howto/variable.rst
@@ -0,0 +1,72 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Managing Variables
+==================
+
+Variables are a generic way to store and retrieve arbitrary content or
+settings as a simple key value store within Airflow. Variables can be
+listed, created, updated and deleted from the UI (``Admin -> Variables``),
+code or CLI.
+
+.. image:: ../img/variable_hidden.png
+
+See the :ref:`Variables Concepts <concepts:variables>` documentation for
+more information.
+
+Storing Variables in Environment Variables
+------------------------------------------
+
+.. versionadded:: 1.10.10
+
+Airflow Variables can also be created and managed using Environment Variables. The environment variable
+naming convention is :envvar:`AIRFLOW_VAR_{VARIABLE_NAME}`, all uppercase.
+So if your variable key is ``FOO`` then the variable name should be ``AIRFLOW_VAR_FOO``.
+
+For example,
+
+.. code-block:: bash
+
+ export AIRFLOW_VAR_FOO=BAR
+
+ # To use JSON, store them as JSON strings
+ export AIRFLOW_VAR_FOO_BAZ='{"hello":"world"}'
+
+You can use them in your DAGs as:
+
+.. code-block:: python
+
+ from airflow.models import Variable
+ foo = Variable.get("foo")
+ foo_json = Variable.get("foo_baz", deserialize_json=True)
+
+.. note::
+
+ Single underscores surround ``VAR``. This is in contrast with the way ``airflow.cfg``
+ parameters are stored, where double underscores surround the config section name.
+ Variables set using Environment Variables would not appear in the Airflow UI but you will
+ be able to use it in your DAG file.
+
+Securing Variables
+------------------
+
+Airflow uses `Fernet <https://github.com/fernet/spec/>`__ to encrypt variables stored in the metastore database.
+It guarantees that without the encryption password, content cannot be manipulated or read
+without the key. For information on configuring Fernet, look at :ref:`security/fernet`.
+
+In addition to retrieving variables from environment variables or the metastore database, you can enable
+a secrets backend to retrieve variables. For more details see :doc:`secrets-backend/index`