You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by gi...@apache.org on 2020/08/26 18:03:10 UTC
[airflow-site] branch asf-site updated: Update asf-site to output
generated at bead826
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 134ef1b Update asf-site to output generated at bead826
134ef1b is described below
commit 134ef1b71feddbf8a69728db2d59872c96ecf0b8
Author: kaxil <ka...@gmail.com>
AuthorDate: Wed Aug 26 18:02:58 2020 +0000
Update asf-site to output generated at bead826
---
_gen/indexes/en/blog-index.json | 2 +-
_gen/indexes/en/blog-posts.json | 2 +-
blog/airflow-1.10.10/index.html | 35 +++-
.../index.html | 219 ++++++++++++++++-----
blog/airflow-1.10.8-1.10.9/index.html | 35 +++-
blog/airflow-survey/index.html | 4 +-
blog/announcing-new-website/index.html | 4 +-
blog/apache-airflow-for-newcomers/index.html | 8 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
blog/index.html | 31 +++
blog/index.xml | 170 ++++++++++++++++
.../index.html | 4 +-
blog/tags/release/index.html | 33 +++-
blog/tags/release/index.xml | 172 +++++++++++++++-
index.html | 30 +--
index.xml | 170 ++++++++++++++++
search/index.html | 4 +-
sitemap.xml | 123 ++++++------
tags/index.html | 2 +-
tags/index.xml | 2 +-
use-cases/adobe/index.html | 4 +-
use-cases/big-fish-games/index.html | 4 +-
use-cases/dish/index.html | 4 +-
use-cases/experity/index.html | 4 +-
use-cases/onefootball/index.html | 4 +-
28 files changed, 922 insertions(+), 164 deletions(-)
diff --git a/_gen/indexes/en/blog-index.json b/_gen/indexes/en/blog-index.json
index 1d49821..e2ee9a9 100644
--- a/_gen/indexes/en/blog-index.json
+++ b/_gen/indexes/en/blog-index.json
@@ -1 +1 @@
-{"version":"2.3.8","fields":["title","description","author","content","tags","url"],"fieldVectors":[["title/It's a \"Breeze\" to develop Apache Airflow",[0,1.962,1,0.736,2,0.15,3,0.389,4,0.474]],["description/It's a \"Breeze\" to develop Apache Airflow",[2,0.158,4,0.323,5,1.337,6,0.631,7,1.337,8,0.792,9,0.631,10,0.631,11,0.631,12,0.502,13,1.337,14,0.502,15,1.337,16,0.086]],["author/It's a \"Breeze\" to develop Apache Airflow",[11,0.942,17,1.184]],["content/It's a \"Breeze\" to develop Ap [...]
\ No newline at end of file
+{"version":"2.3.8","fields":["title","description","author","content","tags","url"],"fieldVectors":[["title/It's a \"Breeze\" to develop Apache Airflow",[0,2.014,1,0.802,2,0.092,3,0.384,4,0.474]],["description/It's a \"Breeze\" to develop Apache Airflow",[2,0.097,4,0.323,5,1.373,6,0.675,7,1.373,8,0.834,9,0.675,10,0.675,11,0.675,12,0.547,13,1.373,14,0.547,15,1.373,16,0.136]],["author/It's a \"Breeze\" to develop Apache Airflow",[11,1.023,17,1.265]],["content/It's a \"Breeze\" to develop A [...]
\ No newline at end of file
diff --git a/_gen/indexes/en/blog-posts.json b/_gen/indexes/en/blog-posts.json
index 5b9ded8..cedabb1 100644
--- a/_gen/indexes/en/blog-posts.json
+++ b/_gen/indexes/en/blog-posts.json
@@ -1 +1 @@
-[{"content":"## The story behind the Airflow Breeze tool\nInitially, we started contributing to this fantastic open-source project [Apache Airflow] with a team of three which then grew to five. When we kicked it off a year ago, I realized pretty soon where the biggest bottlenecks and areas for improvement in terms of productivity were. Even with the help of our client, who provided us with a “homegrown” development environment it took us literally days to set it up and learn some basics. [...]
\ No newline at end of file
+[{"content":"## The story behind the Airflow Breeze tool\nInitially, we started contributing to this fantastic open-source project [Apache Airflow] with a team of three which then grew to five. When we kicked it off a year ago, I realized pretty soon where the biggest bottlenecks and areas for improvement in terms of productivity were. Even with the help of our client, who provided us with a “homegrown” development environment it took us literally days to set it up and learn some basics. [...]
\ No newline at end of file
diff --git a/blog/airflow-1.10.10/index.html b/blog/airflow-1.10.10/index.html
index 8ec5b2e..38ec1e9 100644
--- a/blog/airflow-1.10.10/index.html
+++ b/blog/airflow-1.10.10/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-04-09T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.10">
<meta itemprop="description" content="We are happy to present Apache Airflow 1.10.10">
<meta itemprop="datePublished" content="2020-04-09T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="1143">
@@ -777,6 +777,37 @@ objc[1873]: +[__NSPlaceholderDate initialize] may have been in progress in anoth
</div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Tue, Aug 25, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Apache Airflow 1.10.12</p>
+ <p class="box-event__blogpost--author">Kaxil Naik</p>
+ <p class="box-event__blogpost--description">We are happy to present Apache Airflow 1.10.12</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-1.10.12/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
<span class="bodytext__medium--brownish-grey box-event__blogpost--date">Sun, Feb 23, 2020</span>
</div>
<p class="box-event__blogpost--header">Apache Airflow 1.10.8 & 1.10.9</p>
diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.12/index.html
similarity index 84%
copy from blog/airflow-1.10.8-1.10.9/index.html
copy to blog/airflow-1.10.12/index.html
index 9bd0268..559dda4 100644
--- a/blog/airflow-1.10.8-1.10.9/index.html
+++ b/blog/airflow-1.10.12/index.html
@@ -29,21 +29,21 @@
<meta name="msapplication-TileImage" content="/favicons/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
-<title>Apache Airflow 1.10.8 & 1.10.9 | Apache Airflow</title><meta property="og:title" content="Apache Airflow 1.10.8 & 1.10.9" />
-<meta property="og:description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow." />
+<title>Apache Airflow 1.10.12 | Apache Airflow</title><meta property="og:title" content="Apache Airflow 1.10.12" />
+<meta property="og:description" content="We are happy to present Apache Airflow 1.10.12" />
<meta property="og:type" content="article" />
-<meta property="og:url" content="/blog/airflow-1.10.8-1.10.9/" />
+<meta property="og:url" content="/blog/airflow-1.10.12/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:published_time" content="2020-02-23T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
-<meta itemprop="name" content="Apache Airflow 1.10.8 & 1.10.9">
-<meta itemprop="description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow.">
+<meta property="article:published_time" content="2020-08-25T00:00:00+00:00" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta itemprop="name" content="Apache Airflow 1.10.12">
+<meta itemprop="description" content="We are happy to present Apache Airflow 1.10.12">
-<meta itemprop="datePublished" content="2020-02-23T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
-<meta itemprop="wordCount" content="437">
+<meta itemprop="datePublished" content="2020-08-25T00:00:00+00:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
+<meta itemprop="wordCount" content="911">
@@ -51,8 +51,8 @@
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:image" content="/images/feature-image.png"/>
-<meta name="twitter:title" content="Apache Airflow 1.10.8 & 1.10.9"/>
-<meta name="twitter:description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow."/>
+<meta name="twitter:title" content="Apache Airflow 1.10.12"/>
+<meta name="twitter:description" content="We are happy to present Apache Airflow 1.10.12"/>
<script type="application/javascript">
@@ -80,7 +80,7 @@ if (!doNotTrack) {
crossorigin="anonymous"></script>
-<meta name="description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow." />
+<meta name="description" content="We are happy to present Apache Airflow 1.10.12" />
@@ -439,9 +439,9 @@ if (!doNotTrack) {
</div>
- <span class="bodytext__medium--brownish-grey">Sun, Feb 23, 2020</span>
+ <span class="bodytext__medium--brownish-grey">Tue, Aug 25, 2020</span>
</div>
- <p class="blogpost-content__metadata--title">Apache Airflow 1.10.8 & 1.10.9</p>
+ <p class="blogpost-content__metadata--title">Apache Airflow 1.10.12</p>
<div class="blogpost-content__metadata--author">
<span class="blogpost-content__metadata--author">
Kaxil Naik
@@ -485,55 +485,131 @@ if (!doNotTrack) {
</a>
</div>
- <p class="blogpost-content__metadata--description">We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow.</p>
+ <p class="blogpost-content__metadata--description">We are happy to present Apache Airflow 1.10.12</p>
</div>
<div class="markdown-content">
-<p>Airflow 1.10.8 contains 160 commits since 1.10.7 and includes 4 new features, 42 improvements, 36 bug fixes, and several doc changes.</p>
-
-<p>We released 1.10.9 on the same day as one of the Flask dependencies (Werkzeug) released 1.0 which broke Airflow 1.10.8.</p>
+<p>Airflow 1.10.12 contains 113 commits since 1.10.11 and includes 5 new features, 23 improvements, 23 bug fixes,
+and several doc changes.</p>
<p><strong>Details</strong>:</p>
<ul>
-<li><strong>PyPI</strong>: <a href="https://pypi.org/project/apache-airflow/1.10.9/" target="_blank">https://pypi.org/project/apache-airflow/1.10.9/</a></li>
-<li><strong>Docs</strong>: <a href="https://airflow.apache.org/docs/1.10.9/" target="_blank">https://airflow.apache.org/docs/1.10.9/</a></li>
-<li><strong>Changelog (1.10.8)</strong>: <a href="http://airflow.apache.org/docs/1.10.8/changelog.html#airflow-1-10-8-2020-01-07" target="_blank">http://airflow.apache.org/docs/1.10.8/changelog.html#airflow-1-10-8-2020-01-07</a></li>
-<li><strong>Changelog (1.10.9)</strong>: <a href="http://airflow.apache.org/docs/1.10.9/changelog.html#airflow-1-10-9-2020-02-10" target="_blank">http://airflow.apache.org/docs/1.10.9/changelog.html#airflow-1-10-9-2020-02-10</a></li>
+<li><strong>PyPI</strong>: <a href="https://pypi.org/project/apache-airflow/1.10.12/" target="_blank">https://pypi.org/project/apache-airflow/1.10.12/</a></li>
+<li><strong>Docs</strong>: <a href="https://airflow.apache.org/docs/1.10.12/" target="_blank">https://airflow.apache.org/docs/1.10.12/</a></li>
+<li><strong>Changelog</strong>: <a href="http://airflow.apache.org/docs/1.10.12/changelog.html" target="_blank">http://airflow.apache.org/docs/1.10.12/changelog.html</a></li>
</ul>
+<p><strong>Airflow 1.10.11 has breaking changes with respect to
+KubernetesExecutor & KubernetesPodOperator so I recommend users to directly upgrade to Airflow 1.10.12 instead</strong>.</p>
+
<p>Some of the noteworthy new features (user-facing) are:</p>
<ul>
-<li><a href="https://github.com/apache/airflow/pull/6489" target="_blank">Add tags to DAGs and use it for filtering in the UI (RBAC only)</a></li>
-<li><a href="http://airflow.apache.org/docs/1.10.9/executor/debug.html" target="_blank">New Executor: DebugExecutor for Local debugging from your IDE</a></li>
-<li><a href="https://github.com/apache/airflow/pull/7281" target="_blank">Allow passing conf in “Add DAG Run” (Triggered Dags) view</a></li>
-<li><a href="https://github.com/apache/airflow/pull/7038" target="_blank">Allow dags to run for future execution dates for manually triggered DAGs (only if <code>schedule_interval=None</code>)</a></li>
-<li><a href="https://airflow.apache.org/docs/1.10.9/configurations-ref.html" target="_blank">Dedicated page in documentation for all configs in airflow.cfg</a></li>
+<li><a href="https://github.com/apache/airflow/pull/8560" target="_blank">Allow defining custom XCom class</a></li>
+<li><a href="https://github.com/apache/airflow/pull/9645" target="_blank">Get Airflow configs with sensitive data from Secret Backends</a></li>
+<li><a href="https://github.com/apache/airflow/pull/10282" target="_blank">Add AirflowClusterPolicyViolation support to Airflow local settings</a></li>
+</ul>
+
+<h3 id="allow-defining-custom-xcom-class">Allow defining Custom XCom class</h3>
+
+<p>Until Airflow 1.10.11, the XCom data was only stored in Airflow Metadatabase. From Airflow 1.10.12, users
+would be able to define custom XCom classes. This will allow users to transfer larger data between tasks.
+An example here would be to store XCom in S3 or GCS Bucket if the size of data that needs to be stored is larger
+than <code>XCom.MAX_XCOM_SIZE</code> (48 KB).</p>
+
+<p><strong>PR</strong>: <a href="https://github.com/apache/airflow/pull/8560" target="_blank">https://github.com/apache/airflow/pull/8560</a></p>
+
+<h3 id="get-airflow-configs-with-sensitive-data-from-secret-backends">Get Airflow configs with sensitive data from Secret Backends</h3>
+
+<p>Users would be able to get the following Airflow configs from Secrets Backend like Hashicorp Vault:</p>
+
+<ul>
+<li><code>sql_alchemy_conn</code> in [core] section</li>
+<li><code>fernet_key</code> in [core] section</li>
+<li><code>broker_url</code> in [celery] section</li>
+<li><code>flower_basic_auth</code> in [celery] section</li>
+<li><code>result_backend</code> in [celery] section</li>
+<li><code>password</code> in [atlas] section</li>
+<li><code>smtp_password</code> in [smtp] section</li>
+<li><code>bind_password</code> in [ldap] section</li>
+<li><code>git_password</code> in [kubernetes] section</li>
</ul>
-<h3 id="add-tags-to-dags-and-use-it-for-filtering-in-the-ui">Add tags to DAGs and use it for filtering in the UI</h3>
+<p>Further improving Airflow’s Secret Management story, from Airflow 1.10.12, users don’t need to hardcode
+the <strong>sensitive</strong> config value in airflow.cfg nor then need to use an Environment variable to set this config.</p>
+
+<p>For example, the metadata database connection string can either be set in airflow.cfg like this:</p>
+
+<pre><code class="language-ini">[core]
+sql_alchemy_conn_secret = sql_alchemy_conn
+</code></pre>
-<p>In order to filter DAGs (e.g by team), you can add tags in each dag. The filter is saved in a cookie and can be reset by the reset button.</p>
+<p>This will retrieve config option from the set Secret Backends.</p>
-<p>For example:</p>
+<p>As you can see you just need to add a <code>_secret</code> suffix at the end of the actual config option
+and the value needs to be the <strong>key</strong> which the Secrets backend will look for.</p>
-<p>In your Dag file, pass a list of tags you want to add to DAG object:</p>
+<p>Similarly, <code>_secret</code> config options can also be set using a corresponding environment variable. For example:</p>
-<pre><code class="language-python">dag = DAG(
- dag_id='example_dag_tag',
- schedule_interval='0 0 * * *',
- tags=['example']
-)
+<pre><code>export AIRFLOW__CORE__SQL_ALCHEMY_CONN_SECRET=sql_alchemy_conn
</code></pre>
-<p><strong>Screenshot</strong>:
-<img src="airflow-dag-tags.png" alt="Add filter by DAG tags" /></p>
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/howto/set-config.html" target="_blank">http://airflow.apache.org/docs/1.10.12/howto/set-config.html</a></p>
+
+<h3 id="add-airflowclusterpolicyviolation-support-to-airflow-local-settings-py">Add AirflowClusterPolicyViolation support to airflow_local_settings.py</h3>
+
+<p>Users can use Cluster Policies to apply cluster-wide checks on Airflow
+tasks. You can raise <a href="http://airflow.apache.org/docs/1.10.12/_api/airflow/exceptions/index.html#airflow.exceptions.AirflowClusterPolicyViolation" target="_blank">AirflowClusterPolicyViolation</a>
+in a policy or task mutation hook to prevent a DAG from being
+imported or prevent a task from being executed if the task is not compliant with
+your check.</p>
+
+<p>These checks are intended to help teams using Airflow to protect against common
+beginner errors that may get past a code reviewer, rather than as technical
+security controls.</p>
-<p><strong>Note</strong>: This feature is only available for the RBAC UI (enabled using <code>rbac=True</code> in <code>[webserver]</code> section in your <code>airflow.cfg</code>).</p>
+<p>For example, don’t run tasks without <code>airflow</code> owners:</p>
+
+<pre><code class="language-python">def task_must_have_owners(task):
+ if not task.owner or task.owner.lower() == conf.get('operators', 'default_owner'):
+ raise AirflowClusterPolicyViolation(
+ 'Task must have non-None non-default owner. Current value: {}'.format(task.owner))
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks" target="_blank">http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks</a></p>
+
+<h3 id="launch-pods-via-yaml-files-when-using-kubernetesexecutor-and-kubernetespodoperator">Launch Pods via YAML files when using KubernetesExecutor and KubernetesPodOperator</h3>
+
+<p>As of 1.10.12, users can launch pods via YAML files instead of passing various configurations.</p>
+
+<p>To allow greater flexibility we have deprecated Airflow’s Pod class and instead now use classes and
+objects from the official Kubernetes API. The POD class will still work but raise a deprecation
+warning. This feature involved a pretty extensive rewrite of all of our pod creation code.</p>
+
+<p>Initially, we were going to hold off on these features until Airflow 2.0. However, we soon
+realized that exposing these features in 1.10.x is crucial in preparing users for the 2.0 release to come.</p>
+
+<p>Details: <a href="https://github.com/apache/airflow/pull/6230" target="_blank">https://github.com/apache/airflow/pull/6230</a> (<a href="https://github.com/apache/airflow/commit/7aa0f472b57985a952a3e3d0a38f1b2535d93413" target="_blank">Backport commit</a>)</p>
+
+<h2 id="updating-guide">Updating Guide</h2>
+
+<p>If you are updating Apache Airflow from a previous version to <code>1.10.12</code>, please take a note of the following:</p>
+
+<ul>
+<li><p>Run <code>airflow upgradedb</code> after <code>pip install -U apache-airflow==1.10.12</code> as <code>1.10.12</code> contains 1 database migration.</p></li>
+
+<li><p>As of airflow 1.10.12, using the <code>airflow.contrib.kubernetes.Pod</code> class in the <code>pod_mutation_hook</code> is now
+deprecated. Instead we recommend that users treat the pod parameter as a <code>kubernetes.client.models.V1Pod</code> object.
+This means that users now have access to the full Kubernetes API when modifying airflow pods for mutating POD.</p></li>
+
+<li><p>Previously, when tasks skipped by SkipMixin (such as <code>BranchPythonOperator</code>, <code>BaseBranchOperator</code> and
+<code>ShortCircuitOperator</code>) are cleared, they execute. Since 1.10.12, when such skipped tasks are cleared,
+they will be skipped again by the newly introduced <code>NotPreviouslySkippedDep</code>.</p></li>
+</ul>
-<h2 id="special-note-deprecations">Special Note / Deprecations</h2>
+<h2 id="special-note">Special Note</h2>
<h3 id="python-2">Python 2</h3>
@@ -544,19 +620,31 @@ Airflow 1.10.* would be the last series to support Python 2.</p>
<h3 id="use-airflow-rbac-ui">Use Airflow RBAC UI</h3>
-<p>Airflow 1.10.9 ships with 2 UIs, the default is non-RBAC Flask-admin based UI and Flask-appbuilder based UI.</p>
+<p>Airflow 1.10.10 ships with 2 UIs, the default is non-RBAC Flask-admin based UI and Flask-appbuilder based UI.</p>
-<p>The Flask-AppBuilder (FAB) based UI is allows Role-based Access Control and has more advanced features compared to
-the legacy Flask-admin based UI. This UI can be enabled by setting <code>rbac=True</code> in <code>[webserver]</code> section in your <code>airflow.cfg</code>.</p>
+<p>The Flask-AppBuilder (FAB) based UI allows Role-based Access Control and has more advanced features compared to
+the legacy Flask-admin based UI. This UI can be enabled by setting <code>rbac=True</code> in <code>[webserver]</code> section in
+your <code>airflow.cfg</code>.</p>
<p>Flask-admin based UI is deprecated and new features won’t be ported to it. This UI will still be the default
for 1.10.* series but would no longer be available from Airflow 2.0</p>
+<h3 id="we-have-moved-to-github-issues">We have moved to Github Issues</h3>
+
+<p>The Airflow Project has moved from <a href="https://issues.apache.org/jira/projects/AIRFLOW/issues" target="_blank">JIRA</a> to
+<a href="https://github.com/apache/airflow/issues" target="_blank">Github</a> for tracking issues.</p>
+
+<p>So if you find any bugs in Airflow 1.10.10 please create a Github Issue for it.</p>
+
<h2 id="list-of-contributors">List of Contributors</h2>
-<p>According to git shortlog, the following people contributed to the 1.10.8 and 1.10.9 release. Thank you to all contributors!</p>
+<p>According to git shortlog, the following people contributed to the 1.10.10 release. Thank you to all contributors!</p>
-<p>Anita Fronczak, Ash Berlin-Taylor, BasPH, Bharat Kashyap, Bharath Palaksha, Bhavika Tekwani, Bjorn Olsen, Brian Phillips, Cooper Gillan, Daniel Cohen, Daniel Imberman, Daniel Standish, Gabriel Eckers, Hossein Torabi, Igor Khrol, Jacob, Jarek Potiuk, Jay, Jiajie Zhong, Jithin Sukumar, Kamil Breguła, Kaxil Naik, Kousuke Saruta, Mustafa Gök, Noël Bardelot, Oluwafemi Sule, Pete DeJoy, QP Hou, Qian Yu, Robin Edwards, Ry Walker, Steven van Rossum, Tomek Urbaszek, Xinbin Huang, Yuen-Kuei Hsu [...]
+<p>Alexander Sutcliffe, Andy, Aneesh Joseph, Ash Berlin-Taylor, Aviral Agrawal, BaoshanGu, Beni Ben zikry,
+Daniel Imberman, Daniel Standish, Danylo Baibak, Ephraim Anierobi, Felix Uellendall, Greg Neiheisel,
+Hartorn, Jacob Ferriero, Jannik F, Jarek Potiuk, Jinhui Zhang, Kamil Breguła, Kaxil Naik, Kurganov,
+Luis Magana, Max Arrich, Pete DeJoy, Sumit Maheshwari, Tomek Urbaszek, Vicken Simonian, Vinnie Guimaraes,
+William Tran, Xiaodong Deng, YI FU, Zikun Zhu, dewaldabrie, pulsar314, retornam, yuqian90</p>
</div>
@@ -614,16 +702,16 @@ for 1.10.* series but would no longer be available from Airflow 2.0</p>
<div class="pager">
- <a href="/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/">
+ <a href="/blog/apache-airflow-for-newcomers/">
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Previous</button>
</a>
- <a href="/blog/airflow-1.10.10/">
+ <a >
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Next</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Next</button>
</a>
</div>
@@ -672,6 +760,37 @@ for 1.10.* series but would no longer be available from Airflow 2.0</p>
</div>
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Sun, Feb 23, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Apache Airflow 1.10.8 & 1.10.9</p>
+ <p class="box-event__blogpost--author">Kaxil Naik</p>
+ <p class="box-event__blogpost--description">We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow.</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-1.10.8-1.10.9/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
</div>
</div>
</div>
@@ -684,7 +803,7 @@ for 1.10.* series but would no longer be available from Airflow 2.0</p>
<div class="base-layout--button">
- <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/blog/airflow-1.10.8-1.10.9/index.md>
+ <a href=https://github.com/apache/airflow-site/edit/master/landing-pages/site/content/en/blog/airflow-1.10.12/index.md>
diff --git a/blog/airflow-1.10.8-1.10.9/index.html b/blog/airflow-1.10.8-1.10.9/index.html
index 9bd0268..bbe84ff 100644
--- a/blog/airflow-1.10.8-1.10.9/index.html
+++ b/blog/airflow-1.10.8-1.10.9/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-02-23T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow 1.10.8 & 1.10.9">
<meta itemprop="description" content="We are happy to present the new 1.10.8 and 1.10.9 releases of Apache Airflow.">
<meta itemprop="datePublished" content="2020-02-23T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="437">
@@ -654,6 +654,37 @@ for 1.10.* series but would no longer be available from Airflow 2.0</p>
</div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Tue, Aug 25, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Apache Airflow 1.10.12</p>
+ <p class="box-event__blogpost--author">Kaxil Naik</p>
+ <p class="box-event__blogpost--description">We are happy to present Apache Airflow 1.10.12</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-1.10.12/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
<span class="bodytext__medium--brownish-grey box-event__blogpost--date">Thu, Apr 9, 2020</span>
</div>
<p class="box-event__blogpost--header">Apache Airflow 1.10.10</p>
diff --git a/blog/airflow-survey/index.html b/blog/airflow-survey/index.html
index 6e2119e..adc4761 100644
--- a/blog/airflow-survey/index.html
+++ b/blog/airflow-survey/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Airflow Survey 2019">
<meta itemprop="description" content="Receiving and adjusting to our users’ feedback is a must. Let’s see who Airflow users are, how they play with it, and what they miss.">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="1775">
diff --git a/blog/announcing-new-website/index.html b/blog/announcing-new-website/index.html
index e9c8899..d760ded 100644
--- a/blog/announcing-new-website/index.html
+++ b/blog/announcing-new-website/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-11T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="New Airflow website">
<meta itemprop="description" content="We are thrilled about our new website!">
<meta itemprop="datePublished" content="2019-12-11T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="282">
diff --git a/blog/apache-airflow-for-newcomers/index.html b/blog/apache-airflow-for-newcomers/index.html
index 992ba10..5edf8b5 100644
--- a/blog/apache-airflow-for-newcomers/index.html
+++ b/blog/apache-airflow-for-newcomers/index.html
@@ -37,14 +37,14 @@ Authoring Workflow in Apache Airflow. Airflow makes it easy to author workflows
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-08-17T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Apache Airflow For Newcomers">
<meta itemprop="description" content="Apache Airflow is a platform to programmatically author, schedule, and monitor workflows. A workflow is a sequence of tasks that processes a set of data. You can think of workflow as the path that describes how tasks go from being undone to done. Scheduling, on the other hand, is the process of planning, controlling, and optimizing when a particular task should be done.
Authoring Workflow in Apache Airflow. Airflow makes it easy to author workflows using python scripts.">
<meta itemprop="datePublished" content="2020-08-17T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="1070">
@@ -687,10 +687,10 @@ and <a href="https://github.com/turbaszek" target="_blank">Tomek</a> for all the
<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Previous</button>
</a>
- <a >
+ <a href="/blog/airflow-1.10.12/">
-<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" disabled>Next</button>
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Next</button>
</a>
</div>
diff --git a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
index 2153077..da26086 100644
--- a/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
+++ b/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="ApacheCon Europe 2019 — Thoughts and Insights by Airflow Committers">
<meta itemprop="description" content="Here come some thoughts by Airflow committers and contributors from the ApacheCon Europe 2019. Get to know the ASF community!">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="138">
diff --git a/blog/documenting-using-local-development-environments/index.html b/blog/documenting-using-local-development-environments/index.html
index 0d73b12..244bca2 100644
--- a/blog/documenting-using-local-development-environments/index.html
+++ b/blog/documenting-using-local-development-environments/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Documenting using local development environment">
<meta itemprop="description" content="The story behind documenting local development environment of Apache Airflow">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="256">
diff --git a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
index 3bad51a..336d58f 100644
--- a/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
+++ b/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/index.html
@@ -37,14 +37,14 @@ About Me I have been writing tech articles on medium as well as my blog for the
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-12-20T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experience in Google Season of Docs 2019 with Apache Airflow">
<meta itemprop="description" content="I came across Google Season of Docs (GSoD) almost by accident, thanks to my extensive HackerNews and Twitter addiction. I was familiar with the Google Summer of Code but not with this program. It turns out it was the inaugural phase. I read the details, and the process felt a lot like GSoC except that this was about documentation.
About Me I have been writing tech articles on medium as well as my blog for the past 1.">
<meta itemprop="datePublished" content="2019-12-20T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="1521">
diff --git a/blog/implementing-stable-api-for-apache-airflow/index.html b/blog/implementing-stable-api-for-apache-airflow/index.html
index 5271b51..d57eae6 100644
--- a/blog/implementing-stable-api-for-apache-airflow/index.html
+++ b/blog/implementing-stable-api-for-apache-airflow/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2020-07-19T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Implementing Stable API for Apache Airflow">
<meta itemprop="description" content="An Outreachy intern's progress report on contributing to Apache Airflow REST API.">
<meta itemprop="datePublished" content="2020-07-19T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="703">
diff --git a/blog/index.html b/blog/index.html
index 3848457..286238b 100644
--- a/blog/index.html
+++ b/blog/index.html
@@ -484,6 +484,37 @@ if (!doNotTrack) {
<div class="tags-container">
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Tue, Aug 25, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Apache Airflow 1.10.12</p>
+ <p class="box-event__blogpost--author">Kaxil Naik</p>
+ <p class="box-event__blogpost--description">We are happy to present Apache Airflow 1.10.12</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-1.10.12/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
<a class="tag" href="/blog/tags/community/">Community</a>
diff --git a/blog/index.xml b/blog/index.xml
index 82338ad..4202ffb 100644
--- a/blog/index.xml
+++ b/blog/index.xml
@@ -14,6 +14,176 @@
<item>
+ <title>Blog: Apache Airflow 1.10.12</title>
+ <link>/blog/airflow-1.10.12/</link>
+ <pubDate>Tue, 25 Aug 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-1.10.12/</guid>
+ <description>
+
+
+
+
+<p>Airflow 1.10.12 contains 113 commits since 1.10.11 and includes 5 new features, 23 improvements, 23 bug fixes,
+and several doc changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<ul>
+<li><strong>PyPI</strong>: <a href="https://pypi.org/project/apache-airflow/1.10.12/" target="_blank">https://pypi.org/project/apache-airflow/1.10.12/</a></li>
+<li><strong>Docs</strong>: <a href="https://airflow.apache.org/docs/1.10.12/" target="_blank">https://airflow.apache.org/docs/1.10.12/</a></li>
+<li><strong>Changelog</strong>: <a href="http://airflow.apache.org/docs/1.10.12/changelog.html" target="_blank">http://airflow.apache.org/docs/1.10.12/changelog.html</a></li>
+</ul>
+
+<p><strong>Airflow 1.10.11 has breaking changes with respect to
+KubernetesExecutor &amp; KubernetesPodOperator so I recommend users to directly upgrade to Airflow 1.10.12 instead</strong>.</p>
+
+<p>Some of the noteworthy new features (user-facing) are:</p>
+
+<ul>
+<li><a href="https://github.com/apache/airflow/pull/8560" target="_blank">Allow defining custom XCom class</a></li>
+<li><a href="https://github.com/apache/airflow/pull/9645" target="_blank">Get Airflow configs with sensitive data from Secret Backends</a></li>
+<li><a href="https://github.com/apache/airflow/pull/10282" target="_blank">Add AirflowClusterPolicyViolation support to Airflow local settings</a></li>
+</ul>
+
+<h3 id="allow-defining-custom-xcom-class">Allow defining Custom XCom class</h3>
+
+<p>Until Airflow 1.10.11, the XCom data was only stored in Airflow Metadatabase. From Airflow 1.10.12, users
+would be able to define custom XCom classes. This will allow users to transfer larger data between tasks.
+An example here would be to store XCom in S3 or GCS Bucket if the size of data that needs to be stored is larger
+than <code>XCom.MAX_XCOM_SIZE</code> (48 KB).</p>
+
+<p><strong>PR</strong>: <a href="https://github.com/apache/airflow/pull/8560" target="_blank">https://github.com/apache/airflow/pull/8560</a></p>
+
+<h3 id="get-airflow-configs-with-sensitive-data-from-secret-backends">Get Airflow configs with sensitive data from Secret Backends</h3>
+
+<p>Users would be able to get the following Airflow configs from Secrets Backend like Hashicorp Vault:</p>
+
+<ul>
+<li><code>sql_alchemy_conn</code> in [core] section</li>
+<li><code>fernet_key</code> in [core] section</li>
+<li><code>broker_url</code> in [celery] section</li>
+<li><code>flower_basic_auth</code> in [celery] section</li>
+<li><code>result_backend</code> in [celery] section</li>
+<li><code>password</code> in [atlas] section</li>
+<li><code>smtp_password</code> in [smtp] section</li>
+<li><code>bind_password</code> in [ldap] section</li>
+<li><code>git_password</code> in [kubernetes] section</li>
+</ul>
+
+<p>Further improving Airflow&rsquo;s Secret Management story, from Airflow 1.10.12, users don&rsquo;t need to hardcode
+the <strong>sensitive</strong> config value in airflow.cfg nor then need to use an Environment variable to set this config.</p>
+
+<p>For example, the metadata database connection string can either be set in airflow.cfg like this:</p>
+
+<pre><code class="language-ini">[core]
+sql_alchemy_conn_secret = sql_alchemy_conn
+</code></pre>
+
+<p>This will retrieve config option from the set Secret Backends.</p>
+
+<p>As you can see you just need to add a <code>_secret</code> suffix at the end of the actual config option
+and the value needs to be the <strong>key</strong> which the Secrets backend will look for.</p>
+
+<p>Similarly, <code>_secret</code> config options can also be set using a corresponding environment variable. For example:</p>
+
+<pre><code>export AIRFLOW__CORE__SQL_ALCHEMY_CONN_SECRET=sql_alchemy_conn
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/howto/set-config.html" target="_blank">http://airflow.apache.org/docs/1.10.12/howto/set-config.html</a></p>
+
+<h3 id="add-airflowclusterpolicyviolation-support-to-airflow-local-settings-py">Add AirflowClusterPolicyViolation support to airflow_local_settings.py</h3>
+
+<p>Users can use Cluster Policies to apply cluster-wide checks on Airflow
+tasks. You can raise <a href="http://airflow.apache.org/docs/1.10.12/_api/airflow/exceptions/index.html#airflow.exceptions.AirflowClusterPolicyViolation" target="_blank">AirflowClusterPolicyViolation</a>
+in a policy or task mutation hook to prevent a DAG from being
+imported or prevent a task from being executed if the task is not compliant with
+your check.</p>
+
+<p>These checks are intended to help teams using Airflow to protect against common
+beginner errors that may get past a code reviewer, rather than as technical
+security controls.</p>
+
+<p>For example, don&rsquo;t run tasks without <code>airflow</code> owners:</p>
+
+<pre><code class="language-python">def task_must_have_owners(task):
+ if not task.owner or task.owner.lower() == conf.get('operators', 'default_owner'):
+ raise AirflowClusterPolicyViolation(
+ 'Task must have non-None non-default owner. Current value: {}'.format(task.owner))
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks" target="_blank">http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks</a></p>
+
+<h3 id="launch-pods-via-yaml-files-when-using-kubernetesexecutor-and-kubernetespodoperator">Launch Pods via YAML files when using KubernetesExecutor and KubernetesPodOperator</h3>
+
+<p>As of 1.10.12, users can launch pods via YAML files instead of passing various configurations.</p>
+
+<p>To allow greater flexibility we have deprecated Airflow&rsquo;s Pod class and instead now use classes and
+objects from the official Kubernetes API. The POD class will still work but raise a deprecation
+warning. This feature involved a pretty extensive rewrite of all of our pod creation code.</p>
+
+<p>Initially, we were going to hold off on these features until Airflow 2.0. However, we soon
+realized that exposing these features in 1.10.x is crucial in preparing users for the 2.0 release to come.</p>
+
+<p>Details: <a href="https://github.com/apache/airflow/pull/6230" target="_blank">https://github.com/apache/airflow/pull/6230</a> (<a href="https://github.com/apache/airflow/commit/7aa0f472b57985a952a3e3d0a38f1b2535d93413" target="_blank">Backport commit</a>)</p>
+
+<h2 id="updating-guide">Updating Guide</h2>
+
+<p>If you are updating Apache Airflow from a previous version to <code>1.10.12</code>, please take a note of the following:</p>
+
+<ul>
+<li><p>Run <code>airflow upgradedb</code> after <code>pip install -U apache-airflow==1.10.12</code> as <code>1.10.12</code> contains 1 database migration.</p></li>
+
+<li><p>As of airflow 1.10.12, using the <code>airflow.contrib.kubernetes.Pod</code> class in the <code>pod_mutation_hook</code> is now
+deprecated. Instead we recommend that users treat the pod parameter as a <code>kubernetes.client.models.V1Pod</code> object.
+This means that users now have access to the full Kubernetes API when modifying airflow pods for mutating POD.</p></li>
+
+<li><p>Previously, when tasks skipped by SkipMixin (such as <code>BranchPythonOperator</code>, <code>BaseBranchOperator</code> and
+<code>ShortCircuitOperator</code>) are cleared, they execute. Since 1.10.12, when such skipped tasks are cleared,
+they will be skipped again by the newly introduced <code>NotPreviouslySkippedDep</code>.</p></li>
+</ul>
+
+<h2 id="special-note">Special Note</h2>
+
+<h3 id="python-2">Python 2</h3>
+
+<p>Python 2 has reached end of its life on Jan 2020. Airflow Master no longer supports Python 2.
+Airflow 1.10.* would be the last series to support Python 2.</p>
+
+<p>We strongly recommend users to use Python &gt;= 3.6</p>
+
+<h3 id="use-airflow-rbac-ui">Use Airflow RBAC UI</h3>
+
+<p>Airflow 1.10.10 ships with 2 UIs, the default is non-RBAC Flask-admin based UI and Flask-appbuilder based UI.</p>
+
+<p>The Flask-AppBuilder (FAB) based UI allows Role-based Access Control and has more advanced features compared to
+the legacy Flask-admin based UI. This UI can be enabled by setting <code>rbac=True</code> in <code>[webserver]</code> section in
+your <code>airflow.cfg</code>.</p>
+
+<p>Flask-admin based UI is deprecated and new features won&rsquo;t be ported to it. This UI will still be the default
+for 1.10.* series but would no longer be available from Airflow 2.0</p>
+
+<h3 id="we-have-moved-to-github-issues">We have moved to Github Issues</h3>
+
+<p>The Airflow Project has moved from <a href="https://issues.apache.org/jira/projects/AIRFLOW/issues" target="_blank">JIRA</a> to
+<a href="https://github.com/apache/airflow/issues" target="_blank">Github</a> for tracking issues.</p>
+
+<p>So if you find any bugs in Airflow 1.10.10 please create a Github Issue for it.</p>
+
+<h2 id="list-of-contributors">List of Contributors</h2>
+
+<p>According to git shortlog, the following people contributed to the 1.10.10 release. Thank you to all contributors!</p>
+
+<p>Alexander Sutcliffe, Andy, Aneesh Joseph, Ash Berlin-Taylor, Aviral Agrawal, BaoshanGu, Beni Ben zikry,
+Daniel Imberman, Daniel Standish, Danylo Baibak, Ephraim Anierobi, Felix Uellendall, Greg Neiheisel,
+Hartorn, Jacob Ferriero, Jannik F, Jarek Potiuk, Jinhui Zhang, Kamil Breguła, Kaxil Naik, Kurganov,
+Luis Magana, Max Arrich, Pete DeJoy, Sumit Maheshwari, Tomek Urbaszek, Vicken Simonian, Vinnie Guimaraes,
+William Tran, Xiaodong Deng, YI FU, Zikun Zhu, dewaldabrie, pulsar314, retornam, yuqian90</p>
+
+ </description>
+ </item>
+
+ <item>
<title>Blog: Apache Airflow For Newcomers</title>
<link>/blog/apache-airflow-for-newcomers/</link>
<pubDate>Mon, 17 Aug 2020 00:00:00 +0000</pubDate>
diff --git a/blog/its-a-breeze-to-develop-apache-airflow/index.html b/blog/its-a-breeze-to-develop-apache-airflow/index.html
index 9b3b9ac..a2b8042 100644
--- a/blog/its-a-breeze-to-develop-apache-airflow/index.html
+++ b/blog/its-a-breeze-to-develop-apache-airflow/index.html
@@ -36,13 +36,13 @@
<meta property="og:image" content="/images/feature-image.png" />
<meta property="article:published_time" content="2019-11-22T00:00:00+00:00" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="It's a "Breeze" to develop Apache Airflow">
<meta itemprop="description" content="A Principal Software Engineer's journey to developer productivity. Learn how Jarek and his team speeded up and simplified Airflow development for the community.">
<meta itemprop="datePublished" content="2019-11-22T00:00:00+00:00" />
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="127">
diff --git a/blog/tags/release/index.html b/blog/tags/release/index.html
index 4790c39..51c5058 100644
--- a/blog/tags/release/index.html
+++ b/blog/tags/release/index.html
@@ -37,7 +37,7 @@
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2020-04-09T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="og:updated_time" content="2020-08-25T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="release">
<meta itemprop="description" content="Platform created by the community to programmatically author, schedule and monitor workflows.">
@@ -497,6 +497,37 @@ if (!doNotTrack) {
</div>
+ <span class="bodytext__medium--brownish-grey box-event__blogpost--date">Tue, Aug 25, 2020</span>
+ </div>
+ <p class="box-event__blogpost--header">Apache Airflow 1.10.12</p>
+ <p class="box-event__blogpost--author">Kaxil Naik</p>
+ <p class="box-event__blogpost--description">We are happy to present Apache Airflow 1.10.12</p>
+ <div class="mt-auto">
+ <a href="/blog/airflow-1.10.12/">
+
+
+<button class="btn-hollow btn-blue bodytext__medium--cerulean-blue" >Read more</button>
+
+ </a>
+ </div>
+ </div>
+</div>
+
+ </div>
+
+ <div class="list-item list-item--wide">
+
+
+<div class="card">
+ <div class="box-event__blogpost">
+ <div class="box-event__blogpost--metadata">
+ <div class="tags-container">
+
+
+ <a class="tag" href="/blog/tags/release/">Release</a>
+
+
+ </div>
<span class="bodytext__medium--brownish-grey box-event__blogpost--date">Thu, Apr 9, 2020</span>
</div>
<p class="box-event__blogpost--header">Apache Airflow 1.10.10</p>
diff --git a/blog/tags/release/index.xml b/blog/tags/release/index.xml
index e9304b2..5613a7b 100644
--- a/blog/tags/release/index.xml
+++ b/blog/tags/release/index.xml
@@ -4,7 +4,7 @@
<link>/blog/tags/release/</link>
<description>Recent content in release on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
- <lastBuildDate>Thu, 09 Apr 2020 00:00:00 +0000</lastBuildDate>
+ <lastBuildDate>Tue, 25 Aug 2020 00:00:00 +0000</lastBuildDate>
<atom:link href="/blog/tags/release/index.xml" rel="self" type="application/rss+xml" />
@@ -15,6 +15,176 @@
<item>
+ <title>Blog: Apache Airflow 1.10.12</title>
+ <link>/blog/airflow-1.10.12/</link>
+ <pubDate>Tue, 25 Aug 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-1.10.12/</guid>
+ <description>
+
+
+
+
+<p>Airflow 1.10.12 contains 113 commits since 1.10.11 and includes 5 new features, 23 improvements, 23 bug fixes,
+and several doc changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<ul>
+<li><strong>PyPI</strong>: <a href="https://pypi.org/project/apache-airflow/1.10.12/" target="_blank">https://pypi.org/project/apache-airflow/1.10.12/</a></li>
+<li><strong>Docs</strong>: <a href="https://airflow.apache.org/docs/1.10.12/" target="_blank">https://airflow.apache.org/docs/1.10.12/</a></li>
+<li><strong>Changelog</strong>: <a href="http://airflow.apache.org/docs/1.10.12/changelog.html" target="_blank">http://airflow.apache.org/docs/1.10.12/changelog.html</a></li>
+</ul>
+
+<p><strong>Airflow 1.10.11 has breaking changes with respect to
+KubernetesExecutor &amp; KubernetesPodOperator so I recommend users to directly upgrade to Airflow 1.10.12 instead</strong>.</p>
+
+<p>Some of the noteworthy new features (user-facing) are:</p>
+
+<ul>
+<li><a href="https://github.com/apache/airflow/pull/8560" target="_blank">Allow defining custom XCom class</a></li>
+<li><a href="https://github.com/apache/airflow/pull/9645" target="_blank">Get Airflow configs with sensitive data from Secret Backends</a></li>
+<li><a href="https://github.com/apache/airflow/pull/10282" target="_blank">Add AirflowClusterPolicyViolation support to Airflow local settings</a></li>
+</ul>
+
+<h3 id="allow-defining-custom-xcom-class">Allow defining Custom XCom class</h3>
+
+<p>Until Airflow 1.10.11, the XCom data was only stored in Airflow Metadatabase. From Airflow 1.10.12, users
+would be able to define custom XCom classes. This will allow users to transfer larger data between tasks.
+An example here would be to store XCom in S3 or GCS Bucket if the size of data that needs to be stored is larger
+than <code>XCom.MAX_XCOM_SIZE</code> (48 KB).</p>
+
+<p><strong>PR</strong>: <a href="https://github.com/apache/airflow/pull/8560" target="_blank">https://github.com/apache/airflow/pull/8560</a></p>
+
+<h3 id="get-airflow-configs-with-sensitive-data-from-secret-backends">Get Airflow configs with sensitive data from Secret Backends</h3>
+
+<p>Users would be able to get the following Airflow configs from Secrets Backend like Hashicorp Vault:</p>
+
+<ul>
+<li><code>sql_alchemy_conn</code> in [core] section</li>
+<li><code>fernet_key</code> in [core] section</li>
+<li><code>broker_url</code> in [celery] section</li>
+<li><code>flower_basic_auth</code> in [celery] section</li>
+<li><code>result_backend</code> in [celery] section</li>
+<li><code>password</code> in [atlas] section</li>
+<li><code>smtp_password</code> in [smtp] section</li>
+<li><code>bind_password</code> in [ldap] section</li>
+<li><code>git_password</code> in [kubernetes] section</li>
+</ul>
+
+<p>Further improving Airflow&rsquo;s Secret Management story, from Airflow 1.10.12, users don&rsquo;t need to hardcode
+the <strong>sensitive</strong> config value in airflow.cfg nor then need to use an Environment variable to set this config.</p>
+
+<p>For example, the metadata database connection string can either be set in airflow.cfg like this:</p>
+
+<pre><code class="language-ini">[core]
+sql_alchemy_conn_secret = sql_alchemy_conn
+</code></pre>
+
+<p>This will retrieve config option from the set Secret Backends.</p>
+
+<p>As you can see you just need to add a <code>_secret</code> suffix at the end of the actual config option
+and the value needs to be the <strong>key</strong> which the Secrets backend will look for.</p>
+
+<p>Similarly, <code>_secret</code> config options can also be set using a corresponding environment variable. For example:</p>
+
+<pre><code>export AIRFLOW__CORE__SQL_ALCHEMY_CONN_SECRET=sql_alchemy_conn
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/howto/set-config.html" target="_blank">http://airflow.apache.org/docs/1.10.12/howto/set-config.html</a></p>
+
+<h3 id="add-airflowclusterpolicyviolation-support-to-airflow-local-settings-py">Add AirflowClusterPolicyViolation support to airflow_local_settings.py</h3>
+
+<p>Users can use Cluster Policies to apply cluster-wide checks on Airflow
+tasks. You can raise <a href="http://airflow.apache.org/docs/1.10.12/_api/airflow/exceptions/index.html#airflow.exceptions.AirflowClusterPolicyViolation" target="_blank">AirflowClusterPolicyViolation</a>
+in a policy or task mutation hook to prevent a DAG from being
+imported or prevent a task from being executed if the task is not compliant with
+your check.</p>
+
+<p>These checks are intended to help teams using Airflow to protect against common
+beginner errors that may get past a code reviewer, rather than as technical
+security controls.</p>
+
+<p>For example, don&rsquo;t run tasks without <code>airflow</code> owners:</p>
+
+<pre><code class="language-python">def task_must_have_owners(task):
+ if not task.owner or task.owner.lower() == conf.get('operators', 'default_owner'):
+ raise AirflowClusterPolicyViolation(
+ 'Task must have non-None non-default owner. Current value: {}'.format(task.owner))
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks" target="_blank">http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks</a></p>
+
+<h3 id="launch-pods-via-yaml-files-when-using-kubernetesexecutor-and-kubernetespodoperator">Launch Pods via YAML files when using KubernetesExecutor and KubernetesPodOperator</h3>
+
+<p>As of 1.10.12, users can launch pods via YAML files instead of passing various configurations.</p>
+
+<p>To allow greater flexibility we have deprecated Airflow&rsquo;s Pod class and instead now use classes and
+objects from the official Kubernetes API. The POD class will still work but raise a deprecation
+warning. This feature involved a pretty extensive rewrite of all of our pod creation code.</p>
+
+<p>Initially, we were going to hold off on these features until Airflow 2.0. However, we soon
+realized that exposing these features in 1.10.x is crucial in preparing users for the 2.0 release to come.</p>
+
+<p>Details: <a href="https://github.com/apache/airflow/pull/6230" target="_blank">https://github.com/apache/airflow/pull/6230</a> (<a href="https://github.com/apache/airflow/commit/7aa0f472b57985a952a3e3d0a38f1b2535d93413" target="_blank">Backport commit</a>)</p>
+
+<h2 id="updating-guide">Updating Guide</h2>
+
+<p>If you are updating Apache Airflow from a previous version to <code>1.10.12</code>, please take a note of the following:</p>
+
+<ul>
+<li><p>Run <code>airflow upgradedb</code> after <code>pip install -U apache-airflow==1.10.12</code> as <code>1.10.12</code> contains 1 database migration.</p></li>
+
+<li><p>As of airflow 1.10.12, using the <code>airflow.contrib.kubernetes.Pod</code> class in the <code>pod_mutation_hook</code> is now
+deprecated. Instead we recommend that users treat the pod parameter as a <code>kubernetes.client.models.V1Pod</code> object.
+This means that users now have access to the full Kubernetes API when modifying airflow pods for mutating POD.</p></li>
+
+<li><p>Previously, when tasks skipped by SkipMixin (such as <code>BranchPythonOperator</code>, <code>BaseBranchOperator</code> and
+<code>ShortCircuitOperator</code>) are cleared, they execute. Since 1.10.12, when such skipped tasks are cleared,
+they will be skipped again by the newly introduced <code>NotPreviouslySkippedDep</code>.</p></li>
+</ul>
+
+<h2 id="special-note">Special Note</h2>
+
+<h3 id="python-2">Python 2</h3>
+
+<p>Python 2 has reached end of its life on Jan 2020. Airflow Master no longer supports Python 2.
+Airflow 1.10.* would be the last series to support Python 2.</p>
+
+<p>We strongly recommend users to use Python &gt;= 3.6</p>
+
+<h3 id="use-airflow-rbac-ui">Use Airflow RBAC UI</h3>
+
+<p>Airflow 1.10.10 ships with 2 UIs, the default is non-RBAC Flask-admin based UI and Flask-appbuilder based UI.</p>
+
+<p>The Flask-AppBuilder (FAB) based UI allows Role-based Access Control and has more advanced features compared to
+the legacy Flask-admin based UI. This UI can be enabled by setting <code>rbac=True</code> in <code>[webserver]</code> section in
+your <code>airflow.cfg</code>.</p>
+
+<p>Flask-admin based UI is deprecated and new features won&rsquo;t be ported to it. This UI will still be the default
+for 1.10.* series but would no longer be available from Airflow 2.0</p>
+
+<h3 id="we-have-moved-to-github-issues">We have moved to Github Issues</h3>
+
+<p>The Airflow Project has moved from <a href="https://issues.apache.org/jira/projects/AIRFLOW/issues" target="_blank">JIRA</a> to
+<a href="https://github.com/apache/airflow/issues" target="_blank">Github</a> for tracking issues.</p>
+
+<p>So if you find any bugs in Airflow 1.10.10 please create a Github Issue for it.</p>
+
+<h2 id="list-of-contributors">List of Contributors</h2>
+
+<p>According to git shortlog, the following people contributed to the 1.10.10 release. Thank you to all contributors!</p>
+
+<p>Alexander Sutcliffe, Andy, Aneesh Joseph, Ash Berlin-Taylor, Aviral Agrawal, BaoshanGu, Beni Ben zikry,
+Daniel Imberman, Daniel Standish, Danylo Baibak, Ephraim Anierobi, Felix Uellendall, Greg Neiheisel,
+Hartorn, Jacob Ferriero, Jannik F, Jarek Potiuk, Jinhui Zhang, Kamil Breguła, Kaxil Naik, Kurganov,
+Luis Magana, Max Arrich, Pete DeJoy, Sumit Maheshwari, Tomek Urbaszek, Vicken Simonian, Vinnie Guimaraes,
+William Tran, Xiaodong Deng, YI FU, Zikun Zhu, dewaldabrie, pulsar314, retornam, yuqian90</p>
+
+ </description>
+ </item>
+
+ <item>
<title>Blog: Apache Airflow 1.10.10</title>
<link>/blog/airflow-1.10.10/</link>
<pubDate>Thu, 09 Apr 2020 00:00:00 +0000</pubDate>
diff --git a/index.html b/index.html
index 824f8ee..99a16d1 100644
--- a/index.html
+++ b/index.html
@@ -1226,12 +1226,12 @@ if (!doNotTrack) {
<div id="integrations-container" class="list-items">
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#azure-microsoft-azure">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Cloud Storage (GCS)</span>
+ <span class="box-event__integration--name">Azure Files</span>
</div>
</div>
@@ -1239,12 +1239,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#service-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Cloud Speech-to-Text</span>
+ <span class="box-event__integration--name">Databricks</span>
</div>
</div>
@@ -1252,12 +1252,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#asf-apache-software-foundation">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Datastore</span>
+ <span class="box-event__integration--name">Apache Sqoop</span>
</div>
</div>
@@ -1265,12 +1265,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#aws-amazon-web-services">
+ <a class="list-item" href="/docs/stable/integration.html#software-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">AWS Glue Catalog</span>
+ <span class="box-event__integration--name">Samba</span>
</div>
</div>
@@ -1278,12 +1278,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
+ <a class="list-item" href="/docs/stable/integration.html#azure-microsoft-azure">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Cloud SQL</span>
+ <span class="box-event__integration--name">Azure Data Lake Storage</span>
</div>
</div>
@@ -1291,12 +1291,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#azure-microsoft-azure">
+ <a class="list-item" href="/docs/stable/integration.html#software-integrations">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Azure Data Lake Storage</span>
+ <span class="box-event__integration--name">Celery</span>
</div>
</div>
@@ -1304,12 +1304,12 @@ if (!doNotTrack) {
- <a class="list-item" href="/docs/stable/integration.html#service-integrations">
+ <a class="list-item" href="/docs/stable/integration.html#gcp-google-cloud-platform">
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Salesforce</span>
+ <span class="box-event__integration--name">Cloud Storage (GCS)</span>
</div>
</div>
@@ -1322,7 +1322,7 @@ if (!doNotTrack) {
<div class="card">
<div class="box-event box-event__integration">
- <span class="box-event__integration--name">Bigtable</span>
+ <span class="box-event__integration--name">Kubernetes Engine</span>
</div>
</div>
diff --git a/index.xml b/index.xml
index 9bef47a..23ed03a 100644
--- a/index.xml
+++ b/index.xml
@@ -13,6 +13,176 @@
<item>
+ <title>Blog: Apache Airflow 1.10.12</title>
+ <link>/blog/airflow-1.10.12/</link>
+ <pubDate>Tue, 25 Aug 2020 00:00:00 +0000</pubDate>
+
+ <guid>/blog/airflow-1.10.12/</guid>
+ <description>
+
+
+
+
+<p>Airflow 1.10.12 contains 113 commits since 1.10.11 and includes 5 new features, 23 improvements, 23 bug fixes,
+and several doc changes.</p>
+
+<p><strong>Details</strong>:</p>
+
+<ul>
+<li><strong>PyPI</strong>: <a href="https://pypi.org/project/apache-airflow/1.10.12/" target="_blank">https://pypi.org/project/apache-airflow/1.10.12/</a></li>
+<li><strong>Docs</strong>: <a href="https://airflow.apache.org/docs/1.10.12/" target="_blank">https://airflow.apache.org/docs/1.10.12/</a></li>
+<li><strong>Changelog</strong>: <a href="http://airflow.apache.org/docs/1.10.12/changelog.html" target="_blank">http://airflow.apache.org/docs/1.10.12/changelog.html</a></li>
+</ul>
+
+<p><strong>Airflow 1.10.11 has breaking changes with respect to
+KubernetesExecutor &amp; KubernetesPodOperator so I recommend users to directly upgrade to Airflow 1.10.12 instead</strong>.</p>
+
+<p>Some of the noteworthy new features (user-facing) are:</p>
+
+<ul>
+<li><a href="https://github.com/apache/airflow/pull/8560" target="_blank">Allow defining custom XCom class</a></li>
+<li><a href="https://github.com/apache/airflow/pull/9645" target="_blank">Get Airflow configs with sensitive data from Secret Backends</a></li>
+<li><a href="https://github.com/apache/airflow/pull/10282" target="_blank">Add AirflowClusterPolicyViolation support to Airflow local settings</a></li>
+</ul>
+
+<h3 id="allow-defining-custom-xcom-class">Allow defining Custom XCom class</h3>
+
+<p>Until Airflow 1.10.11, the XCom data was only stored in Airflow Metadatabase. From Airflow 1.10.12, users
+would be able to define custom XCom classes. This will allow users to transfer larger data between tasks.
+An example here would be to store XCom in S3 or GCS Bucket if the size of data that needs to be stored is larger
+than <code>XCom.MAX_XCOM_SIZE</code> (48 KB).</p>
+
+<p><strong>PR</strong>: <a href="https://github.com/apache/airflow/pull/8560" target="_blank">https://github.com/apache/airflow/pull/8560</a></p>
+
+<h3 id="get-airflow-configs-with-sensitive-data-from-secret-backends">Get Airflow configs with sensitive data from Secret Backends</h3>
+
+<p>Users would be able to get the following Airflow configs from Secrets Backend like Hashicorp Vault:</p>
+
+<ul>
+<li><code>sql_alchemy_conn</code> in [core] section</li>
+<li><code>fernet_key</code> in [core] section</li>
+<li><code>broker_url</code> in [celery] section</li>
+<li><code>flower_basic_auth</code> in [celery] section</li>
+<li><code>result_backend</code> in [celery] section</li>
+<li><code>password</code> in [atlas] section</li>
+<li><code>smtp_password</code> in [smtp] section</li>
+<li><code>bind_password</code> in [ldap] section</li>
+<li><code>git_password</code> in [kubernetes] section</li>
+</ul>
+
+<p>Further improving Airflow&rsquo;s Secret Management story, from Airflow 1.10.12, users don&rsquo;t need to hardcode
+the <strong>sensitive</strong> config value in airflow.cfg nor then need to use an Environment variable to set this config.</p>
+
+<p>For example, the metadata database connection string can either be set in airflow.cfg like this:</p>
+
+<pre><code class="language-ini">[core]
+sql_alchemy_conn_secret = sql_alchemy_conn
+</code></pre>
+
+<p>This will retrieve config option from the set Secret Backends.</p>
+
+<p>As you can see you just need to add a <code>_secret</code> suffix at the end of the actual config option
+and the value needs to be the <strong>key</strong> which the Secrets backend will look for.</p>
+
+<p>Similarly, <code>_secret</code> config options can also be set using a corresponding environment variable. For example:</p>
+
+<pre><code>export AIRFLOW__CORE__SQL_ALCHEMY_CONN_SECRET=sql_alchemy_conn
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/howto/set-config.html" target="_blank">http://airflow.apache.org/docs/1.10.12/howto/set-config.html</a></p>
+
+<h3 id="add-airflowclusterpolicyviolation-support-to-airflow-local-settings-py">Add AirflowClusterPolicyViolation support to airflow_local_settings.py</h3>
+
+<p>Users can use Cluster Policies to apply cluster-wide checks on Airflow
+tasks. You can raise <a href="http://airflow.apache.org/docs/1.10.12/_api/airflow/exceptions/index.html#airflow.exceptions.AirflowClusterPolicyViolation" target="_blank">AirflowClusterPolicyViolation</a>
+in a policy or task mutation hook to prevent a DAG from being
+imported or prevent a task from being executed if the task is not compliant with
+your check.</p>
+
+<p>These checks are intended to help teams using Airflow to protect against common
+beginner errors that may get past a code reviewer, rather than as technical
+security controls.</p>
+
+<p>For example, don&rsquo;t run tasks without <code>airflow</code> owners:</p>
+
+<pre><code class="language-python">def task_must_have_owners(task):
+ if not task.owner or task.owner.lower() == conf.get('operators', 'default_owner'):
+ raise AirflowClusterPolicyViolation(
+ 'Task must have non-None non-default owner. Current value: {}'.format(task.owner))
+</code></pre>
+
+<p>More details: <a href="http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks" target="_blank">http://airflow.apache.org/docs/1.10.12/concepts.html#cluster-policies-for-custom-task-checks</a></p>
+
+<h3 id="launch-pods-via-yaml-files-when-using-kubernetesexecutor-and-kubernetespodoperator">Launch Pods via YAML files when using KubernetesExecutor and KubernetesPodOperator</h3>
+
+<p>As of 1.10.12, users can launch pods via YAML files instead of passing various configurations.</p>
+
+<p>To allow greater flexibility we have deprecated Airflow&rsquo;s Pod class and instead now use classes and
+objects from the official Kubernetes API. The POD class will still work but raise a deprecation
+warning. This feature involved a pretty extensive rewrite of all of our pod creation code.</p>
+
+<p>Initially, we were going to hold off on these features until Airflow 2.0. However, we soon
+realized that exposing these features in 1.10.x is crucial in preparing users for the 2.0 release to come.</p>
+
+<p>Details: <a href="https://github.com/apache/airflow/pull/6230" target="_blank">https://github.com/apache/airflow/pull/6230</a> (<a href="https://github.com/apache/airflow/commit/7aa0f472b57985a952a3e3d0a38f1b2535d93413" target="_blank">Backport commit</a>)</p>
+
+<h2 id="updating-guide">Updating Guide</h2>
+
+<p>If you are updating Apache Airflow from a previous version to <code>1.10.12</code>, please take a note of the following:</p>
+
+<ul>
+<li><p>Run <code>airflow upgradedb</code> after <code>pip install -U apache-airflow==1.10.12</code> as <code>1.10.12</code> contains 1 database migration.</p></li>
+
+<li><p>As of airflow 1.10.12, using the <code>airflow.contrib.kubernetes.Pod</code> class in the <code>pod_mutation_hook</code> is now
+deprecated. Instead we recommend that users treat the pod parameter as a <code>kubernetes.client.models.V1Pod</code> object.
+This means that users now have access to the full Kubernetes API when modifying airflow pods for mutating POD.</p></li>
+
+<li><p>Previously, when tasks skipped by SkipMixin (such as <code>BranchPythonOperator</code>, <code>BaseBranchOperator</code> and
+<code>ShortCircuitOperator</code>) are cleared, they execute. Since 1.10.12, when such skipped tasks are cleared,
+they will be skipped again by the newly introduced <code>NotPreviouslySkippedDep</code>.</p></li>
+</ul>
+
+<h2 id="special-note">Special Note</h2>
+
+<h3 id="python-2">Python 2</h3>
+
+<p>Python 2 has reached end of its life on Jan 2020. Airflow Master no longer supports Python 2.
+Airflow 1.10.* would be the last series to support Python 2.</p>
+
+<p>We strongly recommend users to use Python &gt;= 3.6</p>
+
+<h3 id="use-airflow-rbac-ui">Use Airflow RBAC UI</h3>
+
+<p>Airflow 1.10.10 ships with 2 UIs, the default is non-RBAC Flask-admin based UI and Flask-appbuilder based UI.</p>
+
+<p>The Flask-AppBuilder (FAB) based UI allows Role-based Access Control and has more advanced features compared to
+the legacy Flask-admin based UI. This UI can be enabled by setting <code>rbac=True</code> in <code>[webserver]</code> section in
+your <code>airflow.cfg</code>.</p>
+
+<p>Flask-admin based UI is deprecated and new features won&rsquo;t be ported to it. This UI will still be the default
+for 1.10.* series but would no longer be available from Airflow 2.0</p>
+
+<h3 id="we-have-moved-to-github-issues">We have moved to Github Issues</h3>
+
+<p>The Airflow Project has moved from <a href="https://issues.apache.org/jira/projects/AIRFLOW/issues" target="_blank">JIRA</a> to
+<a href="https://github.com/apache/airflow/issues" target="_blank">Github</a> for tracking issues.</p>
+
+<p>So if you find any bugs in Airflow 1.10.10 please create a Github Issue for it.</p>
+
+<h2 id="list-of-contributors">List of Contributors</h2>
+
+<p>According to git shortlog, the following people contributed to the 1.10.10 release. Thank you to all contributors!</p>
+
+<p>Alexander Sutcliffe, Andy, Aneesh Joseph, Ash Berlin-Taylor, Aviral Agrawal, BaoshanGu, Beni Ben zikry,
+Daniel Imberman, Daniel Standish, Danylo Baibak, Ephraim Anierobi, Felix Uellendall, Greg Neiheisel,
+Hartorn, Jacob Ferriero, Jannik F, Jarek Potiuk, Jinhui Zhang, Kamil Breguła, Kaxil Naik, Kurganov,
+Luis Magana, Max Arrich, Pete DeJoy, Sumit Maheshwari, Tomek Urbaszek, Vicken Simonian, Vinnie Guimaraes,
+William Tran, Xiaodong Deng, YI FU, Zikun Zhu, dewaldabrie, pulsar314, retornam, yuqian90</p>
+
+ </description>
+ </item>
+
+ <item>
<title>Blog: Apache Airflow For Newcomers</title>
<link>/blog/apache-airflow-for-newcomers/</link>
<pubDate>Mon, 17 Aug 2020 00:00:00 +0000</pubDate>
diff --git a/search/index.html b/search/index.html
index 9b9e9c7..fa32f56 100644
--- a/search/index.html
+++ b/search/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/search/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Search Results">
<meta itemprop="description" content="">
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="0">
diff --git a/sitemap.xml b/sitemap.xml
index a90c9a3..940d201 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -4,207 +4,212 @@
<url>
<loc>/docs/overview/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/beds/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/configuring-ponycopters/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/getting-started/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/examples/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/launching-ponycopters/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/multi-bear/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/porridge/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/concepts/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/task/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/tutorial2/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tutorials/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/reference/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/contribution-guidelines/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
- <loc>/blog/apache-airflow-for-newcomers/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <loc>/blog/airflow-1.10.12/</loc>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
- <loc>/blog/tags/community/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <loc>/blog/tags/release/</loc>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/tags/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
+ </url>
+
+ <url>
+ <loc>/blog/apache-airflow-for-newcomers/</loc>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
+ </url>
+
+ <url>
+ <loc>/blog/tags/community/</loc>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/announcements/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/implementing-stable-api-for-apache-airflow/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/rest-api/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.10/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
- </url>
-
- <url>
- <loc>/blog/tags/release/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-1.10.8-1.10.9/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/documentation/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/experience-in-google-season-of-docs-2019-with-apache-airflow/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/airflow-survey/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/announcing-new-website/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/survey/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/users/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/apache-con-europe-2019-thoughts-and-insights-by-airflow-committers/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/tags/development/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/documenting-using-local-development-environments/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/its-a-breeze-to-develop-apache-airflow/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/getting-started/example-page/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/reference/parameter-reference/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/docs/tasks/ponycopters/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/adobe/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/big-fish-games/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/blog/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
@@ -213,62 +218,62 @@
<url>
<loc>/community/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/dish/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/ecosystem/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/experity/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/install/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/meetups/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/onefootball/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/privacy-notice/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/roadmap/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/search/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
<url>
<loc>/use-cases/</loc>
- <lastmod>2020-08-25T21:38:49+01:00</lastmod>
+ <lastmod>2020-08-26T18:59:09+01:00</lastmod>
</url>
</urlset>
\ No newline at end of file
diff --git a/tags/index.html b/tags/index.html
index d53e221..ae68fcf 100644
--- a/tags/index.html
+++ b/tags/index.html
@@ -37,7 +37,7 @@
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="og:updated_time" content="2020-08-17T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="og:updated_time" content="2020-08-25T00:00:00+00:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Tags">
<meta itemprop="description" content="Platform created by the community to programmatically author, schedule and monitor workflows.">
diff --git a/tags/index.xml b/tags/index.xml
index 5eef421..b1ab28e 100644
--- a/tags/index.xml
+++ b/tags/index.xml
@@ -4,7 +4,7 @@
<link>/tags/</link>
<description>Recent content in Tags on Apache Airflow</description>
<generator>Hugo -- gohugo.io</generator>
- <lastBuildDate>Mon, 17 Aug 2020 00:00:00 +0000</lastBuildDate>
+ <lastBuildDate>Tue, 25 Aug 2020 00:00:00 +0000</lastBuildDate>
<atom:link href="/tags/index.xml" rel="self" type="application/rss+xml" />
diff --git a/use-cases/adobe/index.html b/use-cases/adobe/index.html
index 40f6926..0bb1fd3 100644
--- a/use-cases/adobe/index.html
+++ b/use-cases/adobe/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/adobe/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Adobe">
<meta itemprop="description" content="What was the problem? Modern big data platforms need sophisticated data pipelines connecting to many backend services enabling complex workflows. These workflows need to be deployed, monitored, and run either on regular schedules or triggered by external events. Adobe Experience Platform component services architected and built an orchestration service to enable their users to author, schedule, and monitor complex hierarchical (including sequential a [...]
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="251">
diff --git a/use-cases/big-fish-games/index.html b/use-cases/big-fish-games/index.html
index dafc927..a598c05 100644
--- a/use-cases/big-fish-games/index.html
+++ b/use-cases/big-fish-games/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/big-fish-games/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Big Fish Games">
<meta itemprop="description" content="What was the problem? The main challenge is the lack of standardized ETL workflow orchestration tools. PowerShell and Python-based ETL frameworks built in-house are currently used for scheduling and running analytical workloads. However, there is no web UI through which we can monitor these workflows and it requires additional effort to maintain this framework. These scheduled jobs based on external dependencies are not well suited to modern Big Data [...]
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="336">
diff --git a/use-cases/dish/index.html b/use-cases/dish/index.html
index 181bd4c..502b5a5 100644
--- a/use-cases/dish/index.html
+++ b/use-cases/dish/index.html
@@ -35,12 +35,12 @@
<meta property="og:url" content="/use-cases/dish/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Dish">
<meta itemprop="description" content="What was the problem? We faced increasing complexity managing lengthy crontabs with scheduling being an issue, this required carefully planning timing due to resource constraints, usage patterns, and especially custom code needed for retry logic. In the last case, having to verify success of previous jobs and/or steps prior to running the next. Furthermore, time to results is important, but we were increasingly relying on buffers for processing, wher [...]
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="249">
diff --git a/use-cases/experity/index.html b/use-cases/experity/index.html
index 6dc8412..0df2762 100644
--- a/use-cases/experity/index.html
+++ b/use-cases/experity/index.html
@@ -36,13 +36,13 @@ How did Apache Airflow help to solve this problem? Ultimately we decided flexibl
<meta property="og:url" content="/use-cases/experity/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Experity">
<meta itemprop="description" content="What was the problem? We had to deploy our complex, flagship app to multiple nodes in multiple ways. This required tasks to communicate across Windows nodes and coordinate timing perfectly. We did not want to buy an expensive enterprise scheduling tool and needed ultimate flexibility.
How did Apache Airflow help to solve this problem? Ultimately we decided flexible, multi-node, DAG capable tooling was key and airflow was one of the few tools that fit that bill.">
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="191">
diff --git a/use-cases/onefootball/index.html b/use-cases/onefootball/index.html
index 1317e21..fff5e66 100644
--- a/use-cases/onefootball/index.html
+++ b/use-cases/onefootball/index.html
@@ -36,13 +36,13 @@ On top of that, new data tools appear each month: third party data sources, clou
<meta property="og:url" content="/use-cases/onefootball/" />
<meta property="og:image" content="/images/feature-image.png" />
-<meta property="article:modified_time" content="2020-08-25T21:38:49+01:00" /><meta property="og:site_name" content="Apache Airflow" />
+<meta property="article:modified_time" content="2020-08-26T18:59:09+01:00" /><meta property="og:site_name" content="Apache Airflow" />
<meta itemprop="name" content="Onefootball">
<meta itemprop="description" content="What was the problem? With millions of daily active users, managing the complexity of data engineering at Onefootball is a constant challenge. Lengthy crontabs, multiplication of custom API clients, erosion of confidence in the analytics served, increasing heroism (“only one person can solve this issue”). Those are the challenges that most teams face unless they consciously invest in their tools and processes.
On top of that, new data tools appear each month: third party data sources, cloud providers solutions, different storage technologies… Managing all those integrations is costly and brittle, especially for small data engineering teams that are trying to do more with less.">
-<meta itemprop="dateModified" content="2020-08-25T21:38:49+01:00" />
+<meta itemprop="dateModified" content="2020-08-26T18:59:09+01:00" />
<meta itemprop="wordCount" content="294">